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FOREWORD 


This  Annual  Report  presents  the  accomplishments  during  the 
period  of  l  June  1972  to  31  May  1973.  The  principal  investigator  is 
Dr.  D.  C.  Lai  and  co-principal  investigator  is  Dr.  T.  Kailath.  Since 
the  beginning  of  this  project,  it  has  been  the  result  of  the  collabora¬ 
tive  efforts  of  many  individuals.  Most  of  the  staff  members  contri¬ 
buted  in  the  writing  of  this  report. 


Biocybernetic  Factors  in  Human  Perception  and  Memory 

\ 

\  Abstract 

This  project  is  concerned  with  the  application  of  biocybernetic 
concepts  to  the  problems  of  expanding  human  memory.  In  particular,  the 
goal  of  this  research  is  the  enhancement  of  visual  imagery  as  far  as 
possible  in  the  direction  of  photographic  memory.  Scanpaths  of  the  eye 
during  inspection  of  visual  targets  are  treated  as  indicators  of  the 
brain's  strategy  for  the  intake  of  visual  information.'  This  research 
will  determine  the  features  that  differentiate  scanpaths  associated 
with  superior  imagery  from  scanpaths  associated  with  inferior  imagery. 
Similarly,  the  features  correlated  with  superior  imagery  will  be 
differentiated  from  those  correlated  with  inferior  imagery.  A  computer¬ 
ized  biocybernetic  scheme  will  be- implemented  in  an  attempt  to  generate 
image  enhancement  and  to  train  the  individual  to  exert  greater  voluntary 
control  over  his  own  imagery.  ^  In  this  report,  we  give  an  account  of 
our  accomplishments  in  developing  and  implementing  the  necessary 
techniques  for  the  realization  of  the  biocybernetic  scheme. 

The  implications  of  this  project  for  the  Department  of  Defense 
arise  from  the  possibility  of  devising  new  and  uni.sual  techniques  that 
will  permit  the  development  of  stronger  imagery  in  men  with  normal 
memories.  Since  it  is  definitely  an  asset  for  the  military  man  to  have 
the  ability  to  absorb  information  rapidly  and  to  retain  it  intact  for 
long  periods  of  time,  this  project  will  attempt  to  intensify  the  post¬ 
stimulus  imagery  as  far  as  possible  in  the  direction  of  photographic 
memory  and  to  search  out  those  features  of  successful  recall  strategies 
so  as  to  enhance  the  individual's  power  of  redintegration. 
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I.  INTRODUCTION 


This  research  project  is  concerned  with  the  development  of  the 
biocybernetic  concepts  and  techniques  needed  for  the  analysis  and 
development  of  skills  useful  for  the  control  and  enhancement  of  human 
memory  functions,  particularly  those  related  to  the  "eidetic"  images. 
Specifically,  we  are  concentrating  on  the  problem  of  achieving  bio¬ 
cybernetic  expansion  of  human  visual  memory  through  the  use  of  real¬ 
time  computerized  monitoring  and  feedback  of  cues  that  serve  as  keys 
to  memory  encoding  and  retrieval.  This  approach  is  based  on  the 
observation  that  the  human  nervous  system  depends  heavily  on  spatial 
and  temporal  cues  both  in  the  encoding  and  retrieval  of  memories, 
especially  sensory  images. 

Our  plan  is  to  develop  and  implement  techniques  for  real-time 
monitoring  and  prediction  of  central  nervous  activities  through  the 
electroencephalographic  signals  and  through  the  tracking  of  eye  move¬ 
ments  and  eye  fixations.  We  shall  use  these  techniques  to  determine 
the  temporal  and  spatial  cues  that  serve  as  keys  to  human  memory 
encoding  and  memory  retrieval  by  tracking  of  brain  states  and  eye 
positions.  This  information  is  then  utilized  for  arranging  the  desired 
coincidences  between  various  brain  states,  eye  positions,  and  the 
delivery  of  visual  stimulation.  The  visual  stimuli  are  to  be  pres  <=>d 
in  complex  tachistoscopic  batteries  or  screens  for  either  monopic  or 
dichopic  viewing.  The  eye  movements  and  eye  positions  are  to  be 
measured  by  a  unified  system  so  as  to  obtain  their  optical  and  electro- 
physiological  characteristics.  The  brain  states  are  to  be  monitored 
through  the  EEG  signals.  Combining  these  real-time  monitoring  and 
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prediction  techniques  with  a  feedback  scheme  to  close  the  control  loop; 
we  expect  to  obtain  greater  control  of  image  persistence  and  image 
dissipation.  In  other  words,  we  strive  to  use  the  computer  system  to 
supplement  and  strengthen  those  deficiencies  in  human  memory  that 
ordinarily  result  in  image  dissipation.  We  are  assuming  tentatively 
that  a  superior  pattern  of  visual  inspection  that  results  in  a  superior 
memory  is  more  consistent  and  also  less  probably  of  natural  occurrence 
than  a  visual  inspection  strategy  that  is  less  consistent.  Through  the 
use  of  ieal-time  computer  guidance,  we  shail  attempt  to  steer  the  sub¬ 
ject  toward  improved  encoding  and  decoding  strategies  for  memory.  In 
this  report,  we  shall  describe  our  accomplishments  in  designing, 
developing,  and  implementing  the  techniques  for  reacn  ng  these  goals. 

The  general  schemes  are  described  in  the  next  section. 

We  chose  a  PDP-15  computer  system  manufactured  by  Digital  Equipment 
Corporation  (DEC)  for  its  unique  capabilities  suited  to  real-time  data 
acquisition  and  processing  fo’’  the  realization  of  our  schemes  and 
algorithms.  A  major  effort  was  devoted  to  bringing  up  this  new  PDP-15 
computer  system  to  full  operational  status.  This  required  staff 
familiarization  with  the  computer  as  well  as  the  design  and  implemen¬ 
tation  of  assorted  hardware,  mainly  for  interfacing  various  laboratory 
instruments.  The  computer  system  is  now  being  fully  utilized  for  data 
acquisition,  data  analysis,  and  further  software  development.  Several 
experimental  programs  and  general  graphics  software  have  been  completed. 
These  are  described  in  Section  III. 

A  real-time  monitoring  technique  for  EEG  signals  is  described  in 
Section  IV.  Our  work  on  the  measurement  and  tracking  of  eye  movement 
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is  presented  in  Section  V.  Conclusion  and  future  plans  are  made  in 


Section  VI. 

The  significance  of  this  research  project  for  the  Department  of 
Defense  resides  in  the  possibility  of  devising  new  and  unusual  tech¬ 
niques  that  will  permit  the  training  of  stronger  imagery  in  persons 
with  normal  memories.  Since  it  is  definitely  an  asset  for  the  military 
person  to  have  the  ability  to  absorb  information  rapidly  as  well  as  to 
retain  it  intact  for  long  periods  of  time,  tnis  project  attempts  to 
intensify  the  post-stimulus  imagery  as  far  as  possible  in  the  direction 
of  photographic  memory— and  to  identify  and  seek  out  those  features  for 
successful  strategies  of  retrieval  i;o  as  to  enhance  the  redintegrative 
power  of  the  individual. 
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II.  DESCRIPTION  OF  GENERAL  SCHEMES 


Since  this  project  is  concerned  with  the  development  of  the  bio- 
cybernetic  concepts  and  techniques  necessary  for  the  analysis  and 
training  of  skills  useful  for  the  manipulation  and  control  of  the 
concrete  ("eidetic")  images,  we  plan  to  concentrate  on  those  aspects 
of  concrete  memory  which  could  be  expected  to  make  a  substantial 
contribution  to  practical  volitional  memory  if  they  can  be  brought 
under  stronger  individual  control.  It  ls  imperative  for  us  to  develop 
for  analyzing  some  of  the  basic  strategies  used  in  superior  memorization 
and  to  compare  them  with  strategies  which  result  in  an  inferior 
mnemonic  product.  This  would  involve  the  use  of  advanced  signal 
processing  schemes,  pattern  recognition  concepts,  real-time  monito-ing 
and  prediction,  and  advanced  electronic  devices  to  effect  the  control 
of  stimulus  and  response  variables  in  real  time.  Using  these  tech¬ 
niques,  we  expect  to  generate  frequent  and  sustained  combinations  of 
stimulus-response  relationships  which  have  a  low  probability  of  natural 
occurrence.  Our  general  scheme  can  be  divided  into  three  major  areas: 

(1)  To  develop  and  implement  techniques  for  real-time  monitoring 
of  central  nervous  activities  via  the  analysis  of  electroencephalo¬ 
graph^  signals  and  thus  to  determine  the  optimal  times  for  the 
presentation  of  sensory  stimuli. 

(2)  To  develop  and  implement  techniques  for  real-time  monitoring 
of  eye  fixation  and  eye  movement  and  thus  to  determine  in  the 
visual  field  the  optimal  location  for  sensory  stimulation  during 
impression  and  recall. 
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(2)  To  develop  and  implement  techniques  for  feedback  of  the 
temporal  and  spatial  information  to  produce  the  most  effective 
stimulus-response  relationships  during  impression  and  recall. 


The  first  tw^  basic  areas  necessitate  the  use  of  advanced  signal 
processing  and  pattern  recognition  schemes.  The  last  involves  the  use 
of  cybernetics.  All  of  them  require  real-time  monitoring  and  pre¬ 
diction,  and  the  use  of  advanced  electronic  systems.  The  following 
schemes  are  to  be  implemented. 

Denote  the  sequence  of  parameters  of  interest  measured  or  observed 

at  time  instants  ty  tg . tR  by  Y? . Yr  which  are 

vectors.  These  vectors  represent  the  parameters  derived  from  EEG 
signals  or  eye-position  or  eye-movement  measurement.  For  instance,  in 
the  case  of  eye-position  measurement,  we  have 


(II. 1) 


where  (y^)  and  (y  )  r.encte  the  horizontal  and  vertical  coordinates  of 
n  n 

the  eye  position  at  the  ulme  tR;  in  the  case  of  EEG  signals  we  have 


Y  =  f 
n  n 


(II. 2) 


where  Ar,  fR,  and  4>r  signify  the  amplitude,  frequency,  and  phase, 

respectively.  Let  X^  X,, . X^,  Xr,  .  .  .  be  the  sequence  of 

the  true  parameters  desired  to  be  estimated  or  predicted.  It  then 

depends  on  the  measurement  instruments  that  Y  may  relate  to  X  in  the 

n  n 

following  ways: 
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Y  =  MX  +  N 
n  n  n 


(II. 3) 


y  =  M  X  +  N 
n  n  n  n 


(II. 4) 


Y  =  F[X  ,  t  ]  +  N 
n  n  n  n 


(II. 5) 


where  representing  measurement  errors  and  noises  introduced  is 


random  in  nature.  The  first  expression  shows  that  Y  related  to  X  bv 

n  n  J 


a  linear  transformation  as  specified  by  the  matrix  M  and  an  additive 


observation  or  measurement  error  term  symbolized  by  N  .  The  second 

n 


expression  signifies  that  Y  is  related  to  X  by  the  time-varvine 

n  n  j 


linear  transformation  M  plus  an  observation  error  N  .  The  last  one 

n  n 


denotes  that  Y  is  related  to  X  by  a  system  of  nonlinear  transforma 
n  n 


tions,  plus  the  error  term  For  example,  if  the  observation  devices 


are  transducers,  then  the  measurement  is  made  directly;  i.e. ,  there 


will  be  no  transformations  and 


Y  =IX  +  N  =  X  +  N 
n  n  n  n  n 


(II. 6) 


with  Mn  =  I,  the  Identity  matrix.  In  other  words,  the  matrix  M  or 


F[X  ,t  ]  characterize  the  measurement  instrument, 
n  n 


The  desired  parameter  X  is  generated  by  some  physiological  system 


which  may  be  characterized  by  differential  or  difference  equations  and 


can  thus  be  expressed  as 


X  ,  =  <J>  „  X 

n “/  n-/, n  n 


(II. 7) 


where  n  is  the  so-called  transition  matrix.  In  essence,  the  above 

expression  says  that  since  the  parameter  is  generated  by  some  physiolog¬ 


ical  system  the  past,  present,  and  future  values  can  change  only  with 


certain  structural  constraints  and  are  interrelated.  The  differential 


equation  or  equations  are  a  mere  abstraction  of  these  structural 
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constraints  which  set  the  rules  how  the  parameter  values  should  vary 
with  time. 

Since  any  estimation  or  prediction  is  based  on  past  observed  data, 
it  will  be  convenient  to  write  the  equation  in  such  a  way  that  the 
past  observed  data  will  be  in  the  form  as  the  input  to  the  algorithms 
to  be  developed  and  implemented  by  u^ital  computer.  Considering  the 
fixed-length  nature  of  the  computer  memory,  we  shall  introduce  the 
total  observation  vector  and  error  vector  which  represent  the  past  k 
vectors  up  to  the  present  vector: 


Y 

n 

N 

n 

Yn-1 

Nn-1 

Y(n  r 

• 

• 

• 

and 

N,  \  = 
(n) 

• 

• 

•  f; 

Y 

L  n-k  J 

Nn-k 

Assuming  that  the  measurement  instrument  is  characterized  by  the  matrix 
M  and  combining  Equation  (II. 4),  (II. 7),  and  (II.8),  we  obtain: 


Y 

n 

- 

M 

n 

N 

n 

Yn-1 

M  i 

n-1 

9  , 

n-1 ,  n 

Y  -L 

Vi 

• 

• 

A  -r 

n 

• 

• 

• 

• 

Yn-k 

**  m 

X 

G 

S 

_ 1 

<J>  u 

n-k,  nJ 

LNn-k  J 

Defining  the  matrix 


L 

n 


A 


M 

n 


M 

n- 


1 


Ti¬ 


ll 


M  0 
n-k  n-k,  nJ 


(II. 10) 


we  then  have 


Y 

n 


L  X 
n  n 


+  N 


(n) 


(II. 11) 
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which  relates  the  total  observation  vector  to  the  present  desired 

vector  which  represents  the  parameters  of  interest  to  be  estimated  or 

predicted.  The  matrix  encompasses  all  of  the  a  priori  decisions 

made  for  the  filtering  or  prediction  schemes,  viz. ,  how  and  when  we 

observe  and  what  we  believe  we.  are  observing.  The  task  now  is  to 

estimate  X  or  X  ».  In  the  former  case,  we  would  obtain  an  estimate 
n  n+jt 

of  the  p/esent  parameter  values  X^  based  on  the  past  k  observed  values 

Y_  i,'  W1  »  •  •  •  »  Y  ,  and  the  present  observed  values  Y  .  This 
n-K  n~K+i  n-i  n 

is  called  filtering  or  smoothing.  In  the  latter  case,  we  would  obtain 

an  estimate  of  the  future  parameter  values  X  »  at  the  /th  future 

n4  £ 

instc.its  from  the  present  based  on  the  present  and  past  observed  values. 
This  is  the  so-called  "prediction".  Both  the  estimated  values  of  X 

n 

and  X  *  are  to  be  updated  whenever  a  new  measurement  Y  is  made. 
n+l  n+1 

The  criterion  most  popularly  used  for  obtaining  the  best  estimate  is 
the  so-called  least  square  principle  in  which  the  mean  square  error 
between  the  desired  values  and  the  actual  value  estimated  is  minimized. 
Without  going  through  the  detailed  derivations,  we  shall  write  the 
resulting  best  estimator  in  the  least  squares  sense 

\  ■  [<L»VV]T(n)  • 

We  shall  not  derive  here  the  computational  algorithms  for  the  imple¬ 
mentation  of  this  estimator.  There  are  some  classical  algorithms  in 
existence.  We  are  planning  to  devise  our  own  algorithms  to  suit  our 
needs.  The  least-square  criterion  is  based  on  minimizing  the  summed 
(average)  squared  errors.  In  other  words,  the  algorithm  was  set  up 
in  such  a  way  as  to  minimize  the  sum  of  the  square  of  the  randc  .1 
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components  obtained  on  each  successive  sample  drawn  from  the  observation 
process  without  any  regard  for  the  errors  of  the  individual  components. 
The  weighted  least-square  criterion  is  used  while  each  of  the  errors  of 
the  individual  components  is  minimized.  This  leads  to  the  following 
estimate 


where 


X 


—  m 

-  T  -1  T  -1 

.  =  (L  R,  .  L  )  L  V  \  ’ 

n  |_  n  (n)  n  n  (n) 


Cn) 


(II.  13) 


R 


(n) 


(11.14) 


which  is  the  total  covariance  matrix  of  the  input  errors.  The  filtering 
idea  discussed  above  may  be  summarized  by  the  block  diagram  shown  in 
Figure  II.  1. 


differential  or  error  the  desired 

difference  equation  parameter 

according  to  certain 
criteria 


Figure  II. 1  Block  Diagram  Demonstrating  the 
Basis  for  the  Estimation  or 
Prediction  Scheme 


To  illustrate  the  use  of  the  scheme  described  above,  we  shall 
consider  the  problem  of  measuring  eye  position  and  velocity  of  eye 
movement.  To  simplify  the  example,  we  shall  assume  that  we  are  inter¬ 
ested  only  in  the  motion  along  a  single  axis.  The  observation  vector 
vn  becomes  a  scalar  y  .  Suppose  these  measurements  were  made  every  T 
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seconds  (sampling  interval).  Let  the  most  recent  observed  value  be 
y  .  The  total  observation  vector  is  then 


(II.  15) 


We  shall  use  a  polynomial  [  p  (V)]  as  our  model.  The  variable  V  is 

n 

a  time  index  increasing  positively  with  the  time  whose  origin  is 

located  at  t  =  (n-k)T;  i.e.,  kT  seconds  before  most  recent  observation. 

Figure  II.  2  shows  the  Y-axis  in  relation  with  the  time  axis  and  the 

interpolating  polynomial.  When  a  new  value  y  is  observed  after  T 

n 

seconds,  the  polynomial  is  updated  from  the  new  total  observation 
vector 


(n+1) 


n+1 


n-k+1 


(11.16) 


To  be  more  definite,  let  the  polynomial  be  of  the  first  degree  and 
K  =  3,  and  we  are  interested  in  a  one-step  position  and  velocity 
predictor.  Applying  the  least-square  principle,  but  deleting  the 
details,  we  obtain 


Xn+1  =  Vn  +  Vn-l  +  Vn-2  +  Vn-3 


(11.17) 


*n+l  =  —  (Vn  +  Vn-l  +  Vn-2  +  Vn-3)  (II'18) 

where  the  values  of  the  coefficients  a^ . a^  and  h  , .  .  .  ,bd  can 
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be  obtained  by  appropriately  choosing  the  modeling  polynomial  and  using 
the  least -squares  criterion.  In  the  derivation,  we  have  assumed  the 


fixed-length  nature  of  the  filter  memory.  As  each  new  observation  y 

n 

arrives,  the  previous  ones  are  pushed  down  and  y  ,  is  discarded.  The 

n-4 

latest  one-step  predictions  are  computed  by  the  use  of  (11.17)  and 
(11.18).  The  predictor  stores  and  uses  only  the  most  recent  four 
observed  values,  and  any  which  are  older  than  4T  seconds  are  completely 
wiped  out  or  forgotten.  Figure  II. 3  shows  the  block  diagram  of  our 
scheme. 


Y 

n 


Position 
mod  sled  by 

[pOO]n 

• 

Velocity 
modeled  by 

Position  Ouptput  X 


n+1 


Velocity  Output  X 


n+1 


Figure  11.3  Block  Diagram  Depicting  the  Prediction  Algorithm 

Such  estimators  must  be  computed  simultaneously  and  independently 
for  optical  measurements  and  for  electrooculographic  measurements  of 
eye  position.  These  two  sets  of  measurements  must  be  compared 
frequently  and  combined  into  a  single  best  estimator  derived  from  a 
proper  weighing  of  their  respective  contributions  based  on  other  factors, 
e.g. ,  during  eyelid  closures,  EOG  information  is  obviously  superior  to 
optical  information;  on  the  other  hand,  when  eyelids  are  open,  the 
optical  measurements  can  be  used  as  a  reference  for  nulling  out  EOG 


drift  or  error.  The  scheme  described  above  omits  many  details  and 
refinements.  It  is  intended  solely  to  indicate  the  nature  of  the 
approach. 

The  success  of  the  described  scheme  depends  on  a  good  model  for 
the  psycho-physiological  process.  Inappropriate  modeling  of  the 
physiological  process  is  a  serious  source  of  estimation  error.  The 
above-discussed  scheme  assumes  that  an  appropriate  model  for  the 
physiological  process  is  known.  The  modeling  problem,  as  we  all  know, 

is  by  itself  a  major  research  topic.  This  modeling  problem  may  be 
circumvented  by  the  use  of  a  computer-based  model  with  the  ability  to 

mimic  the  actual  physiological  system  as  far  as  the  stimulus-response 
relationship  is  concerned.  We  shall  discuss  such  a  scheme  as  part  of 
our  integrated  man-machine  system. 

Another  assumption,  which  was  made  implicitly,  is  that  the  appro¬ 
priate  choice  of  parameters  has  been  made.  What  parameters  to  measure 
is  again  a  problem  of  major  portions.  Finally,  we  have  also  made  the 
assumption  that  we  can  measure  the  selected  relevant  parameters 
separated  from  other  irrelevant  parameters  or  signals  being  generated 
by  the  physiological  process.  To  achieve  that,  we  plan  to  apply  a 
variety  of  sophisticated  filtering  and  signal  processing  techniques 
and  to  work  in  close  collaboration  with  the  life  scientists  who  are 
skillful  in  making  the  various  eye-pointing  measurements. 

As  mentioned  before,  the  success  of  the  estimate  or  prediction  of 
the  psycho-physiological  parameters  hinges  on  an  appropriate  mathematical 
model.  Since  neither  the  characteristics  cf  the  physiological  process 
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are  completely  known  nor  the  characterization  of  the  physiological 
process  in  mathematical  terms  is  always  possible.  To  circumvent  this 
problem,  we  shall  devise  a  computer-based  model  which  is  continually 
updated  by  a  machine  learning  process.  The  model  is  to  be  realized  by 
an  algorithm  which  relates  a  set  of  stimuli,  responses,  and  internal 
states.  We  will  then  use  this  computer-based  model  to  predict  (forward 
time  analysis)  the  sort  of  stimuli  which  should  be  used  to  the  machine 
in  order  to  produce  the  desired  future  model  responses.  In  conjunction 
with  this  computer-based  model,  we  plan  to  realize  an  optimal  control 
scheme  for  delivery  sequences  of  sensory  stimulation  conditionally 
related  to  eye  position  and  brain  state,  and  thus  to  explore  system¬ 
atically  their  relation  to  memory  recall.  To  facilitate  discussion 
we  have  simplified  our  integrated  control  scheme  and  depict  it  in 
Figure  II. 4.  We  use  the  EEG  signal  (Y^).^  and  the  eye-movement  measure¬ 
ment  (Y^)g  as  our  availabj.-.  outputs.  These  responses  are  closely 
controlled  since  the  models  mimic  or  are  similar  to  the  actual  physiol¬ 
ogic^  process  as  far  as  the  stimulus-response  relation  is  concerned. 
The  boxes  in  the  block  diagram  of  Figure  II.  4  are  labeled  according  to 
their  functions.  All  except  the  interface  electronic  system  can  be 
implemented  on  the  PDP-15  computer. 


I 


Eye-movement  measurement  (Y  ) 

n  2 


Response 


Figure  II. 4  A  Control  Scheme  for 
Memory  Tracking  or  Training 
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III.  COMPUTER  ORGANIZATION  AND  PROGRAMMING 


The  effort  required  to  get  an  intermediate-sized  computer  facility 
running  should  not  be  underestimated.  In  this  section,  we  describe 
our  research  facilities  and  our  effort  in  designing  interface  hardware 
and  programming  in  order  to  realize  our  schemes  and  algorithms.  During 
the  past  year,  one  of  our  major  efforts  was  devoted  to  bringing  up  the 
new  PDP-15  computer  system  to  full  operational  status.  The  computer 
system  is  now  being  fully  utilized  for  both  data  acquisition  and  data 
analysis. 


1.  Hardware 

The  block  diagram  in  Figure  III.l  depicts  our  general  set  up.  The 
PDP-15  computer  system  Is  the  core  of  our  hardware  system  which  consists 
of  the  following  items: 

PDP-15/35  Disk  Operating  Advanced  Monitor  System,  consisting  of: 
24,000  18-bit,  800-ns  core  memory 

-  LA30  DECWriter 

-  PC15  High  Speed  Paper  Tape  Reader/Punch 

-  KE15  Extendeo  Arithmetic  Element 

-  KA15  Automatic  Priority  Interrupt 

-  KW15  Real  Time  Clock 

-  TCI 5  DECTape  Control 

-  TU56  Dual  DECTAPE  Transport 

-  RFj.5  DECdisk  Control 

-  RS09  DECdisk  Drive,  262,144  words 

KM15  Memory  Protect 
KF15  Power  Fail 
FP15  Floating  Point  Processor 
KT15  Memory  Relocation 

TC59D  Magnetic  Tape  Transport  Control  for  up  to  8  TU1QA,  TU10B, 
TU30A,  TU30B  Magnetic  Tape  Transport  Units 
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TU10A  9-Track,  45  ips  Magnetic  Tape  Transport,  800  bpi 

VP15C  Oscilloscope  Display,  VR14  X-Y  Display  Unit  (7"  x  9"  CRT) 
and  Control 

XY15AB  0.0005-Inch  Step  18,000  Steps/Minute  31-inch  drum,  CalComp 
Model  5C3  and  Control 

CR15  300cpm  Reader  and  Control  (Punched  Cards) 

AA15B  Multiplexer  Control  for  up  to  16  Type  AAC3  12-bit  digital- 
to-analog  converter  channels 

AAC3  Digital-to-Analog  Converter,  signal  buffered,  OV  to  +  10V 

AD15  128  Channel  A/D  Converter  (Medium  Speed)  Three-cycle  data 
channel  capacity  with  provision  for  mounting  the  first  32  channels; 
each  4-channel  group  requires  one  BA124 

BA124  Four-Channel  MOS  FET  Multiplex  switch;  one  required  for  each 
4-channel  group 

CSS  18-Bit  Digital  I/O  Interface 
RSX  Software  Package 
RASP  Software 
LP15-F  Line  Printer 

Figure  III.  2  shows  the  physical  set  up  of  the  PDP-15  system  and  its 
peripherals.  All  of  the  interface  hardware,  shown  in  Figure  III. 1 , 
except  the  CSS-18  I/O  interface,  are  designed  and  built  or  to  be  built 
by  us.  A  brief  description  of  these  interface  hardware  is  given  below. 

(a)  General 

A  general  method  for  the  interface  of  the  special  peripheral 
devices  required  for  this  project  was  developed  using  a  special  device 
(the  CSS  18-bit  interface)  provided  by  the  computer  manufacturer.  This 
is  a  bi-directional  interface  package  which  does  all  of  the  address 
decoding  and  timing  functions  required  of  a  device  controller.  Our 
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use  of  several  devices  requires  that  a  device  selector/data  multiplexer 
be  built,  and  then  additional  special  control  (level  shifting,  inverting, 
qtc.),  unique  to  each  device.  Thus  far,  the  special  interface  units 
have  been  designed,  and,  where  noted,  built.  The  device  selector  is 
in  the  planning  stage.  It  is  presently  circumvented  by  paralleling 
several  devices. 

(b)  Tachistoscope 

The  interface  between  the  ICONIX  tachistoscope  and  the  PDP-15  has 
been  completed  and  is  operational.  The  unique  functions  required  were 
level  shifting  and  pulse  generation,  for  a  total  of  six  commands. 

(c)  Random  Access  Slide  Projector 

The  interface  unit  has  been  designed  and  is  currently  under  con¬ 
struction.  Logic  was  required  to  allow  parallel  entry  of  data,  and  to 
properly  time  the  control  signals.  Level  shifting  of  all  command  and 
data  lines  is  also  required. 

(d)  Time-Code  Generator 

The  interface  for  the  tape  search  unit/time  code  generator  is  not 
completed.  The  functions  required  are  timing,  control,  and  inversion 
of  data. 

2.  Software 

To  bring  up  the  PDP-15  computer  to  its  operational  stage  and  to 
fulfill  our  own  special  needs,  we  need  to  generate  software  which  fails 
into  the  following  three  categories: 
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Category  (a)  General  Support  Programs  and  Subroutines 


These  inclc.de  utility  programs  and  library  routines  written  for 
use  by  all  project  personnel.  These  are  extensions  of  the  operating 
system  and  utility  packages  produced  by  the  manufacturer  of  the  computer, 
which  are  required  for  day-to-day  operation. 

Category  (b)  Software  Investigations 

Research  effort  required  for  production  of  functional  programs. 

Most  work  is  concerned  with  the  timing  constraints  on  real-time  system 
operation. 

Category  (c)  Application  Programs 

These  are  programs  which  perform  specific  functions. 

Most  of  the  software  in  categories  (a)  and  (b)  is  completed  since  these 
form  the  basis  for  operation  of  the  system  in  our  application.  The 
software  in  category  (c)  is  an  on-going  effort.  We  shall  describe  our 
accomplishment  in  each  category  separately. 

(a)  General  Support  Programs  and  Subroutines 

The  PDP-15  Real-Time  Executive  (RSX)  was  designed  for  a  general 
scientific  user.  In  order  to  tailor  the  system  to  our  specific  needs, 
certain  modifications  to  the  DEC  software  were  necessary.  Unfortun¬ 
ately,  these  modifications  could  not  be  realized  using  the  RSX  system 
(due  mainly  to  the  large  core  requirements  of  the  resident  monitor). 

We  were,  therefore,  forced  to  use  DEC's  disk-oriented  batch-processing 
monitor  (DOS)  to  update  and  assemble  the  RSX  system. 

The  first  such  modification  was  extending  the  RSX  teletype  handler 
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to  simulate  horizontal  tabulation  on  our  high-speed,  teletype-compatible 
teleprinter.  Unlike  a  teletype,  the  IA30  has  no  hardware  tabulation. 
Additional  core  was  added  within  RSX  to  monitor  the  current  position 
of  the  carriage,  make  note  of  new  read/write  requests,  and  print  blank 
characters  when  necessary  to  achieve  carriage  movement  corresponding 
to  tabulation.  We  are  now  able  to  1)  enter  columnar  data  with  much  less 
mental  effort,  and  2)  reduce  symbolic  file  space  by  at  least  a  factor 
of  two. 

In  order  to  allow  the  system  to  be  "booted  in"  from  disk  without 
using  the  paper  tape  reader,  we  made  a  second  extension  to  RSX.  The 
system  must  be  "booted  in"  to  start  the  system  in  the  morning  and  to 
refresh  memory  following  any  severe  system  failure.  The  delivered 
system  required  a  warm  start  tape  to  be  placed  in  the  paper  tape 
reader  along  with  the  setting  of  certain  console  switches.  By  placing 
a  copy  of  the  warm  start  program  within  the  resident  monitor  (less  than 
20  words)  we  are  now  able  to  warm  start  the  system  by  simply  depressing 
the  START  switch.  Since  paper  tape  is  cumbersome  to  work  with,  this 
saves  time  and  effort. 

In  addition  to  a  powerful  operating  system,  most  computer  users 
need  an  assortment  of  utility  routines  such  as  memory  and  magnetic  tape 
dumps,  graphics  support  (plotter,  CRT),  special  purpose  subroutines, 
magnetic  tape  copy,  etc.  Although  the  DOS  system  supplies  some  utility 
software,  RSX  includes  virtually  none.  A  list  of  the  utility  software 
written  by  us  is  given  below.  The  details  of  this  software  effort  are 
described  in  the  Appendices  A,  B,  and  C, 

Appendix  A:  GS-15  Graphics  System 
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Appendix  B:  DIGIT  Digitizing  System 

Appendix  C:  Miscellaneous  Utility  Software 

CORE/DISK  DUMP 

DEC tape  Dump 

Magnetic  Tape  Dump 

LAS/ISWTCH 

RIPOFF 

REA 

GT132 

EMMXY 

CALDT/CALTD 

CALPLT 

CALCHK 

(b)  Software  Investigation 

Practically  all  of  our  raw  data  are  in  analog  form.  It  is  impera¬ 
tive  that  we  should  have,  in  operation,  a  good  analog-to-digital  conver¬ 
ter  used  to  input  data.  An  I/O  handler  package,  produced  by  the 
manufacturer,  was  available,  and  several  programs  were  written  to  take 
advantage  of  this  handler.  A  measurement  method  was  designed  to  utilize 
a  square  wave  input  (at  1GHZ)  and  an  off-line  data  analysis  program 
which  searches  through  the  data  to  determine  the  consistency  and 
accuracy  of  the  A/D  operation.  It  became  obvious  that  accuracy  of  the 
A/D  was  not  a  significant  cause  of  error,  but  that  millisecond  level 
scheduling  of  the  data  collection  task  was  causing  severe  CPU  utiliza¬ 
tion  problems.  The  strategy  chosen  was  to  tune  both  the  hardware  and 
the  software  for  optimum  A/D  operation. 

Accordingly,  the  standard  I/O  handler  was  replaced  with  a  high 
speed,  less  general  driver  which  reduced  the  need  to  schedule  two  tasks 
virtually  concurrently.  In  addition,  a  hardware  modification  was 
designed  to  allow  the  A/D  to  operate  on  its  own  clock  rather  than 
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requiring  a  CPU  driven  task  to  schedule  each  A/D  operation.  In  combin¬ 
ation,  these  will  allow  a  substantial  incrase  in  the  digitizing  rate 
that  can  be  supported  by  the  system, 

(c)  Applications  Programs 

These  programs  perform  specific  functions  as  specified  by  users. 

So  far,  we  have  developed  and  completed  programs  designed  for  use  in 
acquisition  of  the.  eye-movement  data.  These  can  also  be  used  for  the 
acquisition  of  EEG  data. 

Another  program  (SEER)  is  in  the  development  stage.  This  program 
is  designed  to  use  the  Tektronix  611  CRT  for  image  production  and 
viewing.  Both  of  these  programs  are  described  in  detnil  in  Appendix  D. 
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IV.  REAL-TIME  MONITORING  OF  EEG  SIGNALS 


1,  Introduction 

Stroboscopic  stimulation  of  the  eyes  results  in  changes  of  the 
alpha  activity  of  the  EEG.  When  the  waveform  is  treated  as  a  band- 
limited  signal  it  is  common  to  specify  it  by  parameters  like  amplitude, 
phase,  and  frequency.  Measurement  and  interpretation  of  these  parameters 
(or  any  other  set  of  parameters)  during  stimulation  may  supply  informa¬ 
tion  relevant  to  improvement  of  visual  task  performance. 

Although  spatial  distribution  of  the  electric  field  and  particular¬ 
ly  spatial  coherence  might  be  of  even  greater  interest  because  of  large 
integrating  area  of  scalp  electrodes,  we  have  confined  our  attention  to 
spatially-averaged  temporal  coherence  which  appears  to  be  a  dominant 
factor  in  a  real-time  measurement,  interpretation,  and  prediction  of  the 
above-mentioned  parameters. 

In  Section  2,  we  present  an  analytic  treatment  of  the  temporal 
coherence  problem  that  leads  to  the  basic  scheme  which  we  use  in  order  to  get 
rid  of  the  redundancy  associated  with  the  Q'-waveform;  thi s  redundancy  is , 
incidentally,  an  obstacle  to  the  design  of  a  real-time  digital  system. 

Section  3  presents  the  basic  scheme  and  the  digital  processing  associated 
with  a  representation  of  o-waves  in  a  more  compact  way.  In  Section  A  , 
we  present  the  modified  scheme. 

2.  Averaging  with  Coincident  Peaks 

A  procedure  commonly  used  to  evaluate  parameters  associated  with 
Q'-waveform  is  "averaging  with  coincident  peaks" .  This  method  involves 
averaging  a  large  number  of  "sample-functions"  of  recorded  data  which 
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are  previously  aligned  so  as  to  have  a  local  maximum  at  the  origin. 

The  averaged  signal  looks  like  a  damped  sinusoid,  the  envelope  of 
which  decreases  more  or  less  exponentially  [l],  [2],  The  rate  of 
decrease  serves  as  a  measure  of  the  temporal  coherence,  and  models 
have  been  proposed  to  investigate  the  relationship  between  the  rite 
of  decrease  and  responsiveness  to  visual  input. 

We  show  that  under  certain  conditions  the  averaged  signal  is  a 
good  approximation  to  the  autocorrelation  function  of  the  in-phase  and 
quadrature  components  of  the  analytic  signal  associated  with  the  Q- 
waveform.  We  conclude,  therefore,  that  the  familiar  representat ‘on  of 
the  waveform  by  two  slowly  varying  processes;  viz.,  the  in-phase  and 
quadrature  modulators,  is  applicable  to  the  o-waveform  although  the 
latter,  strictly  speaking,  does  not  fall  into  the  category  of  narrow- 
band  processes. 

The  following  is  a  procedure  for  analyzing  the  envelope  of  a 
waveform  obtained  by  averaging  N  sample  functions  of  band-pass  process 
conditioned  to  have  a  local  maximum  at  t  =  0,  that  is,  averaging  with 
coincident  peaks. 

Let  x^Ct),  i  =  1,  ...  N,  denote  identically  distributed  independent 
band-pass  processes  defined  by 

xi(t)  =  ci(t)  cos  u)Qt  +  si(t)  sin  o^t  (IV.  1) 

where  c^t),  s^t),  Cj(t),  Sj(t)  J»  =  1  •••  N  are  stationary  normal 
processes  with  zero  mean  and 

E[C1 (t)cj (t— t) ]  =  E[si(t)sj(t-t)]  =  r(T)6tJ  (IV. 2) 

E[s1  (t)Cj  (t)]  =  0-v^-i,  j  . 
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It  follows  that  the  autocorrelation  of  x^Ct)  is 

rx(T)  =  r(T)  COS  O^T  . 

Now  define  the  coincident-peaks-sum  process  x(t) 


x(t)  = 


-~r  E  V^V 


where  t^  is  an  instant  of  local  maximum  and  is  given 


c.  (t)  +  O)  s,  (t) 

tanco  t  = - — — 2 — i - 

0  i  ^Qci(t)  -  Si(t) 


t  =  t 


if  the  band-width  is  narrow  with  respect  to  a)Q  we  may  omit  c  (t)  and 
s^t)  and  get 


Si(ti) 


tanVi  -  ^ 


ci(ti)  W 

cosa)oti  *  €l(tl);  sin  Vi  *  e^Tt^y 


where  the  phase  ^(t)  and  the  envelope  e^t)  are  defined  by 


s  (t) 

'“V*1  =^7tT 

Ej  (t)  =  Sj(t)  +  Cj(t) 


we  conclude  that 


Xi(V  =  €i(ti>  COS(030ti  -  =  ei(ti) 


Which  means  that  shifting  the  time  origin  by  t±  is  equivalent  to  the 


initial  conditions: 


(IV.  10) 


S1<0>  ■  S10  a  0;  V0’  -  So  £  0; 


2  2  2  2 

810  *  So  *  S1  <0)  =  So  • 


Now  we  write  x(t)  in  the  form 


N  N 


N 


"  “n"  ^  X1  (t)  =  ~T“  ^  C  (t)  cosod  t  +  -i-  ^  s  (t)  sin  od  t 
i=l  1  i=l  1  0  i=l  1  0 


c(t)  cosoD-t  +  s(t)  sin  mt  =  e(t)  cos  (a>  t  —  4>  (t ) ) 
u  0  o 


(IV. 11) 


where 


N 


c(t)  =  -jj-  ^  c  (t);  s(t)  =  Xrf  s 
i=l  1  w  i=l  1 


(t) 


s(t)  a  2  2 

and  tan4>(t)  =  ~ TT;  €  (t)  =  s  (t)  +  c  (t)  . 
c  (t) 


(IV. 12a-d) 


The  conditional  mean  and  variance  of  the  random  variables  c^t) 
and  si(t)  given  the  initial  conditions  are 


E[si(t)  I  si0'  cio]  =  r(t)s 


iO 


E[ci(t)  |  si()I  ci0]  =  r(t)c 


(IV. 13a-c ) 


iO 


VartS(t)  I  S10'  C10]  *  Var[S(t>  I  s10-  So]  =  rotl'p2<t,] 


where  p(x)  is  the  normalized  autocorrelation 
t  \  r(x) 

P(T)  =  ~'r  (0)  :  (IV. 14) 

Defining  the  N-diraensional  vectors  cQ,  sQ  by  their  respective  components 
Ci0’  Si0'  We  ^ave  ^ue  to  independence 
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E[c(t) 


I  v  so]  - r(t)  •  ~r  E 


cm  =  r<t)c(0)  =  r(t)c 


E[s<t)  1  V  fo]  ■ r(t)  '  ir  E  sio =  r<t>s«»  -  r(t>.0 


(IV.  15a-c) 


(t)  =  Va 


r[c(t)  I  V  fo]  =  Var[s(t>  I  V  =  “F*r0[  ^2(t)] 


Defining  fx(t )  by 


K  (t)  =  E  U(t)  |  V  c^]  *  E2[c(t)|  c0,  s0]  .  r2(t>  £2  (IV,  16) 


we  can  write  the  conditional  distribution  and  the  conditional 
the  envelope  e(t)  as 


mean  of 


’'‘IVV-HX"  ’^17^  (IV.  17) 

E[£l  V  =  fit  ****  (-o)[(l+2o)I0(o)+2oIi(o)]  (IV.  18) 


where  Iq(.)  and  are  the  modified  Bessel  function  of  order  zero 

and  one,  respectively,  and  a  is  given  by 


.2  r  Ne„  2 

o=  _tL  =  _2 _ 2  £  (t) 

.  2  4  2 

4<r  1-P  (t) 


(IV. 19) 


Assuming  that  lim  r(t)  =0,  we  get  the  Rayleigh  distribution  for  the  envelope 
whose  mean  is  decreasing  as  N_/^; 


lim  p[e  I s  ,  c  ]  = 
t-*a  *0  0 


[*?] 


(IV. 17a) 
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(IV. 18a) 


These  expressions  do  not  depend  on  s^,  so  averaging  over  all 
proper  values  of  s^,  will  not  affect  them.  Keeping  in  mind  that 
x^(t)  has  a  local  maximum  at  the  origin,  we  can  write 


lim  p(e) 
t-*® 


lim  E (e) 
t-*® 


(IV. 17b) 


(IV. 18b) 


It  would  be  nice  to  average  out  eQ  from  formulas  (IV. 17)  and  (IV. 18); 
however,  is  a  sum  of  N  independent  random  variables  with  identical 
Rayleigh  distribution. 

We  could  not  obtain  a  closed  expression  for  the  distribution  of  e^, 
but  for  the  large  N  it  could  be  approximated  by  a  Gaussian  distribution. 
Noting  that  N— »®  and/or  t  — ►  0  implies  a-*®,  and  using  asymptotic 
approximations  to  the  Bessel  functions 


I  (x)  ~  (2ttx)  2  Exp[x]  x  »  1  , 


(IV. 20) 


we  get  for  the  conditional  distribution  of  the  envelope 


%  —V  m  4 

p(e  I  c0’  so)ssir  (2tt°'2>  Exp  - (b~j^ 


-!4 


«(2tt<t  )  Exp 


-  (e-pV 

2 


2a 


(IV. 17c) 


The  conditional  mean  is  obtained  by  expanding  (IV. 18)  in  powers  of 

-1 

a 

E[e  |  C0’  S03w  ?  (1  +  “k  )  ‘  e0(1  +  ■  p(t)  (IV.  18c) 


$ 


which  shows  that  for  small  t  rnd/or  large  N  the  conditional  mean  of  the 


envelope  is  proportional  to  the  autocorrelation.  In  order  to  average 
out  we  note  that  for  large  N  the  distributuion  density  p(€q)  is 
concentrated  about  the  mean: 


.■v  ■  ft*)* 


In  that  case,  we  approximate  the  average  over  by  inserting  E(£q) 
for  eQ  and  get 


2  "/a 

p[e(t)]  as  (2-tto-  )  Exp 


-[e-pE(e0)]‘ 


2<r 


-  fir)* 


(IV. 17d) 

p(t) 

(IV. 18d ) 


for  large  N  and/or  small  t. 

Note  that  unless  p(t)  =  0  the  envelope  e(t)  tends  to  p(t)  if  N 
is  sufficiently  large.  For  large  t,  p(t)  tends  to  zero  and  the 
envelope  tends  to  its  constant  asymptote  which  is  N-dependent 


lim  E[e(t)] 

t— ••CO 


•  (sr  ■ 


By  taking  N  to  be  very  large,  the  envelope  converges  to  its  mean  which 
is  proportional  to  the  autocorrelation  r(t). 


The  evaluation  of  the  temporal  coherence  is  reduced  to  the  problem 
of  estimating  the  auto  and  cross-correlation  functions  of  the  two 
slowly  varying  in-,*hase  and  quadrature  modulators.  Moreover,  it  is 
more  efficient  (from  the  system  viewpoint)  to  deal  with  the  slowly  vary¬ 
ing  signals,  provided  the  representation  of  the  a-waveform  by  means 
of  the  two  slowly  varying  signals  is  accurate.  This  observation  led 
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US  to  the  basic  scheme  presented  in  the  next  section. 


3‘  The  Basic  Scheme:  lepresentatlon  and  Monitoring 

>ve  describe  here  a  new  technique  for  real-time  monitoring  and  pre¬ 
dicting  brain  states  via  EEG  signals.  The  EEG  waveform  is  assumed  to 
be  a  narrow-band  process  whose  spectrum  is  centered  around  a  mean 

frequency  which  may  be  slowly  varying.  Designating  the  EEG  signal 
by  f(t),  we  have 

f(t)  =  e(t)  cos[0(t)]  =  e(t)  cos[oj(t)  +^(t)] 

=  s(t)  sin  05  t  +  c (t )  cos  05  t 
°  o 

By  tracking  the  center  frequency  05q,  we  may  represent  the  signal  by 
the  two  processes  s(t)  and  c(t).  If  the  process  is  stationary  whose 
spectrum  is  symmetric  about  then  S(t)  and  c(t)  are  uncorrelated 
so  the  prediction  would  be  easier  to  deal  with. 

To  obtain  the  in-phase  COnponent  c(t)  and  the  quadrature  component 
s(t),  we  use  a  discrete  "phase-locked  loop"  system  described  in  the 
block  diagram  shown  in  Figure  IV. 1,  This  system  tracks  the  center 
frequency  05q  and  resolves  the  EEG  waveform  into  c(t)  and  s(t).  Once 
c(t)  and  s  (t )  are  obtained  for  a  window  width  of  the  EEG  signal,  we 
reduce  the  data  by  taking  the  ffleflh  value,  the  mean  slope,  etc.  This  is 
done  by  fitting  a  linear  segment  to  the  data  in  the  window  through 
recursive  filtering.  The  values  thus  obtained  serve  as  data  vector 
components. 

The  key  to  the  performance  of  this  real-time  monitoring  technique 
is  the  discrete  phase-locked  loop  system  depicted  in  Figure  IV. 1.  This 
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Figure  IV. 1  Resolver  of 
In-phase  and  Quadrature  Signals 

system  includes: 

(a)  Voltage-Controlled  Oscillator 

The  VCO  is  realized  by  generating  the  sine  and  cosine  wave  forms 
through  the  difference  equations 

S  (N)  =  OS(N-I)  +  pC(N-l) 
and  C  (N)  =  oC(N-l)  -  pS(N-l)  . 

Since  the  frequency  and  the  amplitude  are  functions  of  a  and  p,  we 
control  these  parameters  in  such  a  way  as  to  maintain  a  constant  ampli¬ 
tude  and  to  change  the  frequency  in  accordance  with  the  output  of  the 

discriminator.  Specifically,  we  maintain  unit  amplitude  by  making 
2  2 

°  +  P  =  1  and  increase  or  decrease  the  frequency  by  increasing  or 

decreasing  the  parameter  p  according  to  the  output  of  the  discriminator. 

(b)  Frequency  Discriminator 

The  frequency  discriminator  changes  the  frequency  information  in 
s(t)  and  c(t)  to  voltage  which  is  linearly  proportional  to  the 


f  requt  ncy. 


(c)  Low-Pass  Filters 

The  low-pass  filters  are  a  standard  3-pole  Butterworth  filter  with 
a  3-db  bandwidth  about  5  Hz.  The  two  output  signals  c(t)  and  s(t)  of 
the  previous  system  are  then  fed  separately  into  linear  digital  filters 
that  fit  to  each  of  them  the  best  linear  approximation  in  a  window  of 
k  samples  in  the  least  mean  square  sense.  The  width  of  the  window  was 
chosen  to  be  0.15  sec.  The  two  parameters  of  the  linear  approximation, 
viz.  ,  the  mean  and  the  slope,  are  calculated  recursively  by  digital 
filters  known  as  the  "comb"  and  "slope"  filters.  These  filters  are  the 
digital  realization  of  the  transfer  functions  given  by 


C  (Jx)  = 


sinKx 

sinx 


S  (jx)  =  sin[  (K+l)xJ  _  (K-H)cosKx 

4sinx 


2 

4sin  x 


where  C  (jx)  and  S(Jx)  are  the  amplitude  response  of  the  comb  and  slope 
filters,  respectively;  and  x  is  the  normalized  frequency.  The  value 
for  K  is  63  which  corresponds  approximately  to  the  window  width  of 
0. 15  sec.  The  normalized  frequency  x  is  calculated  by 


x 


aT 

2 


where  T  is  the  sampling  period  (2.5  msec),  a'  is  measured  in  radians/ 
second. 

The  phase  characteristics  of  the  two  filters  are  of  the  same  linear 


type 


phase  (u)  =  uT  . 
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Ihcse  filters  are  block  diagrammed  in  Figure  IV. 2. 


Figure  IV. 2  Block  Diagram  for  Linear  Fitting  Filters 

It  should  be  noted  that  the  four  output  of  these  filters  contain  infor 
mation  about  the  frequency  and  may  be  used  as  a  frequency  discriminator 
If  higher  order  "derivatives"  are  needed,  they  can  be  obtained  by  using 
recursive  filters  in  a  similar  fashion. 

4.  The  Modified  System 

The  design  of  the  modified  system  was  motivated  by  the  desire  to 
get  an  accurate  representation  of  the  o-waveform  by  two  slowly  varying 
signals  without  any  phase  delay  or  distortion. 

We  designed  and  implemented  a  digital  real-time  system  for  obtain¬ 
ing  the  representation  of  a  particular  EEC  rhythm  in  its  in-phase  and 
quadrature  components  with  the  constraint  that  the  error  is  zero  when 
synthesized.  In  a  way,  this  system  provides  a  form  of  data  compression. 
The  resultant  representation  can  be  advantageous-! v  employed  for  data 
transmission  and  data  storage  as  well  as  real-time  monitoring  of  either 
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psycho-physiological  states  or  pathological  states.  For  these  appli¬ 
cations,  the  requirement  of  an  error-free  synthesis  or  reproduction  of 
the  EEG  signal  is  imperative. 

A  common  scheme  for  obtaining  the  in-phase  and  quadrature  compo¬ 
nents  of  a  signal  is  to  multiply  vhe  signal  by  an  oscillator  output 

at  quadrature  with  the  center  frequency  f  of  the  signal  and  then 

o 

passing  the  resultants  through  low-pass  filters  in  the  forward  path. 
However,  this  method  does  not  provide  a  faithful  replica  of  the  signal 
when  the  two  components  are  synthesized.  The  error  arises  mainly  from 
the  phase  distortions  of  the  filter  or  the  pure  delays  of  the  non¬ 
recursive  filter.  Our  digital  system  uses,  instead,  an  oscillator 
running  at  twice  the  center  frequency  of  the  signal  and  a  filter  in 
the  feedback  path.  This  filter  does  not  affect  the  reproduction 
fidelity;  however,  it  plays  a  role  in  determining  certain  statistical 
properties  of  the  slowly  varying  components  of  the  resultant  repre¬ 
sentatives.  Hence,  one  may  choose  a  filter  to  minimize  certain  proper¬ 
ties  of  the  outcome  such  as  the  bandwidth,  variance,  etc. 

Let  the  input  signal  in  its  digitized  form  be  y(N).  We  shall 
define  a  two-dimensional  oscillator  vector  H(N)  by 

HT(N)  =  ^cos2TrfQN  sin27tfoN  J  .  (IV.21) 

We  may  write 

y (N)  =  XT(N)H(N)  (IV. 22) 

where  X(N)  is  a  two-dimensional  vector  whose  components  x^(N)  and 
x2(N)  are  the  slowly-varying  components  sought  after.  Note  that  the 
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Equation  (IV. 22)  is  not  unique  since  there  exist  many  X(N)  which  could 

jatisfy  (IV. 22).  For  a  certain  particular  choice,  x  (N)  and  x  (N)  are 

*  2 

related  by  Hilbert  transform.  Let  X(n|n-1)  be  a  function  of  the  past 

values  of  X(N).  For  instance,  X(n|n-1)  could  be  a  linear  combination 

of  X(0),  X(l),  ...,  X(N-l).  Denote  the  difference  between  X(N)  and 
its  X(NjN-l)  by 


E(N)  =  X(N)  -  X(N  N-l ) 


(IV. 23) 


The  norm  of  the  difference  is 


E(N)|!  =  E  (N)E(N) 


V  • 


(IV. 24) 


We  choose  X(N)  by  minimizing  (IV.24)  but  satisfying  (IV.22)  simultane¬ 
ously.  The  solution  thus  obtained  is 


X(N)  -  %  ^1  -  A(N)J  X(n|n-1)  +  H (N ) y  (N ) 


(IV.  25) 


where  I  is  the  identity  matrix  and  A (N)  is  the  matrix  with  its  compo¬ 
nents  given  by  the  double-frequency  oscillator;  i.e.  , 


A  (N)  = 


cos4TTf  N  Sin4rtf  N 
o  o 


sin4TTf  N  -cos4nf  N 
O  o 


(IV. 26 


If  X(n|n-1)  is  the  best  linear  estimator  of  X(N)  based  on  its  past 
history,  then  the  representation  of  the  input  signal  y(N)  by  the  vectoi 
E(N)  corresponds  to  the  innovation  representation  V(N)  with 


2,  „ 

v  (N) 


where  v(N)  =  y(N)  -  j?(N|N-l)  and  ?<n|n-1)  is  the  best  linear 


(IV. 27) 


estimator 


MM 


Ml 


t 

I 


of  y (N)  in  terms  of  its  past  values  y(0),  y(l) . y(N-l). 

The  digital  system  is  depicted  by  a  block  diagram  shown  in  Figure 
IV. 3.  The  oscillator  is  realized  by  a  second  order  difference  equation 
whose  coefficients  are  controlled  by  the  amplitude  regulator  and  the 
output  of  the  frequency  discriminator.  We  have  selected  an  EEG  signal* 
for  analysis  by  our  digital  system.  In  Figure  IV. 4  we  show  both  the 
original  EEG  signal  and  the  EEG  signal  synthesized  from  these  two 
components.  Note  the  high  fidelity  of  the  reproduction.  The  filter 
in  the  feedback  path  was  chosen  to  minimize  the  first  order  difference 
||  X(N)  -  X(N-l)  ||  . 


Figure  IV.  3  Blick  Diagram  of  the  Digital  System 
for  Resolvirg  the  Signal  into  In-phase  and 
Quadrature  Components 


*The  EEG  data  were  furnished  by  Dr.  J.  E.  Anliker  of  NASA-Ames 
Research  Center. 
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Figure  IV. 4  Representation  of  EEG  Signal 

1.  x^ (N) ,  the  in-phase  component. 

2.  x^(N),  the  quadrature  component. 

3.  The  original  EEG  signal. 

4.  The  reproduced  EEG  signal  synthesized 

from  x,  and  x 
1  2 

5.  The  reference  signal. 
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V.  EYE-MOVEMENT  MEASUREMENT  AND  TRACKING 

Measurement  of  Eye  Movement 

Setting  up  an  eye-movement  measurement  and  data  collection  system 
involves  a  variety  of  tasks  such  as  modifying  commercial  equipment  to 
suit  our  needs,  determining  the  limitations  of  this  equipment  as  they 
affect  our  experiments,  calibration  of  equipment,  and  input  of  data 
to  the  PDP-15  computer.  At  present,  our  analysis  of  eye-movement 
includes  computer  programs  to  (1)  plot  the  digitized  eye-movement  data 
on  either  incremental  plotter  or  storage  oscilloscope,  (2)  search 
through  these  data  and  manipulate  them  for  processing  in  various  ways, 
(3)  use  pattern  recognition  techniques  to  determine  "fixations”  in  the 
eye-movement  data,  and  (4)  label  fixations  as  they  are  plotted.  Other 
programs  provide  for  more  detailed  analysis  of  the  eye-movement  data  to 
determine  the  boundary  conditions  between  fixations  and  saccades  with 
the  ultimate  goal  of  predicting  in  real  time  the  transition  from  fix¬ 
ation  to  saccade  and  vice  versa.  The  techniques  for  eye-movement 
measurement  and  eye-movement  data  acquisition,  and  their  analysis  are 
described  separately  as  follows. 

a.  Measurement  Techniques  and  Data  Acquisition 

i.  Measurement  Apparatus:  The  equipment  currently  used  for 
optical  measurements  of  eye-movements  is  a  commercial  unit  manu¬ 
factured  by  Biometrics,  Inc.,  and  dubbed  the  "Eye-Movement  Monitor 
-  Type  SG"  (EMM).  Its  principle  of  operation  involves  the  photo¬ 
electric  scleral  reflection  technique  (shining  infrared  light  onto 
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the  eye  and  picking  up  the  reflected  light  with  pairs  of  photo¬ 
electric  sensors).  Horizontal  movements  are  monitored  from  the 
subject's  right  eye,  and  vertical  movements  are  monitored  from 
the  left  eye.  This  is  based  on  the  assumption  that  both  eyes 
normally  move  together.  To  monitor  horizontal  movements,  the 
sensors  detect  varying  amounts  of  light  reflected  from  the  dark 
portion  (iris)  and  the  white  portion  (sclera)  of  the  eye  as  varying 
areas  of  these  two  regions  move  under  each  sensor  of  the  pair. 

The  output  voltage  of  the  horizontal  channel  of  ‘‘he  tMM  is  pro¬ 
portional  to  the  voltage  difference  between  the  twe  horizontal 
sensors. 

Vertical  movements  are  measured  by  monitoring  the  movement 
of  the  upper  eyelid,  which  is  proportional  to  the  actual  vertical 
movement  of  the  eye  within  a  degree  of  accuracy.  The  vertical 
sensors  are  positioned  so  that  vertical  movement  of  the  lid 
results  in  varying  areas  of  eyelid  passing  under  both  sensors. 

The  output  voltage  of  the  vertical  channel  of  the  EMM  is  proportion¬ 
al  to  the  sum  of  the  voltages  of  the  two  sensors,  which  is  propor¬ 
tional  to  the  area  of  eyelid  reflecting  light  to  the  sensors. 

The  sensors  and  light  sources  are  mounted  on  an  eyeglass 
frame  worn  by  the  subject.  One  source  and  two  sensors  are  mounted 
on  each  of  two  platforms  which  can  be  moved  independently  in  three 
dimensions  to  position  each  sensor-source  group  as  desired.  A 
bothersome  problem  when  adjusting  sensor-source  position  in  any 
one  dimension  is  the  starting  friction  which  must  be  overcome 
because  movement  is  accomplished  by  sliding  a  plastic  sleeve  along 
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a  metal  track.  If  only  a  small  movement  is  desired,  one  tends  to 
overshoot  the  desired  position  when  overcoming  the  starting 
friction.  Frustration  and  wasted  time  rapidly  become  painful 
realities.  Therefore,  a  modification  has  been  made  to  this  sensor- 
source  apparatus  to  facilitate  accurate  and  rapid  adjustment. 
Movement  in  each  direction  is  now  accomplished  by  turning  a  small 
screw  (one  for  each  dimension  of  movement);  the  mechanism  operates 
similar  to  a  micrometer.  This  modil Nation  has  significantly 
reduced  adjustment  and  calibration  problems  and  has  greatly 
increased  efficiency  in  data  collection.  A  picture  of  the 
modified  Biometrics  glasses  is  shown  in  Figure  V.l. 

The  horizontal  and  vertical  output  voltages  from  the  EMM  are 
input  to  separate  analog-to-digital  conversion  channels  on  the 
PDP-15,  digitized  and  stored  on  magnetic  tape  by  a  general 
digitizing  program  or  various  special  calibration  programs. 
Simultaneously,  these  voltages  may  also  be  recorded  on  separate 
channels  of  analog  tape  at  high  speed  so  the  data  can  later  be 
digitized  when  played  back  at  a  speed  slower  than  the  recording 
speed,  effectively  resulting  in  a  high  sampling  rate.  The  sampling 
interval  for  digitizing  the  EMM  data  is  presently  adjustable  up¬ 
wards  from  a  minimum  of  one  millisecond.  This  is  adequate  in  the 
sense  of  the  Nyquist  criterion  since  the  EMM  amplifiers  have  a 
maximum  bandwidth  of  500  Hz.  (2  ms.).  The  basic  limitation  on 
sampling  rate  of  our  computer  system  is  due  to  the  speed  of  the 
magnetic  tape  storage. 

The  remainder  of  the  present  experimental  setup,  shown  in 
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rigure  V.2,  consists  of  a  headrest,  random  access  slide  projector, 
rear  projection  screen,  and  storage  oscilloscope.  With  the  EMM 
glasses  in  place,  the  subject’s  head  is  secured  in  a  headrest 
apparatus  to  minimize  head  movement,  which  creates  problems  that 
will  be  discussed  l'*. ter.  Visual  stimulus  patterns  are  presented 
to  the  subject  .'sing  either  (1)  a  random  access  projector  to 
project  slides  onto  a  rear  projection  screen  in  front  of  the 
subject,  or  (2)  computer-generated  patterns  displayed  on  a  storage 
oscilloscope.  Both  stimulus  systems  are  controlled  by  the  PDP-15 
computer. 

A  parameter  of  concern  is  the  resolution  of  the  EMM.  Consider 
the  eye  fixating  on  a  point  of  the  display.  When  it  fixates  on 
another  point  at  some  distance,  x,  from  the  present  fixation 
point,  it  must  rotate  through  an  angle  equal  to  the  angle  at  the 
eye  (fovea)  rubtended  by  the  cord  connecting  these  two  points. 
Resolution  is  the  smallest  angle  through  which  the  eye  moves  such 
that  the  measurement  output  voltages  of  the  EMM  are  detectably 
different.  Alternatively,  if  the  eye-movement  data  is  plotted,  the 
two  fixations  subtending  this  angle  will  be  distinct.  Resolution 
was  determined  by  presenting  a  series  of  square  dot  matrices, 
each  succeeding  matrix  having  smaller  inter-dot  distances,  to  the 
subject  and  directing  the  subject  to  fixate  these  points.  When 
plotted,  the  distinctiveness  of  adjacent  points  can  be  determined. 
Smaller  and  smaller  inter-dot  distances  were  presented  until  the 
adjacent  plotted  points  could  not  be  distinguished.  From  the 
smallest  distinctive  inter-dot  distance  and  the  distance  from  the 
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plane  of  the  display  to  the  eye,  the  angle  subtended  and  thus  the 
corresponding  angular  movement  Is  determined.  This  is  the 
resolutuion  of  the  EMM  measuring  technique  and  equipment.  Horizon¬ 
tal  resolution  has  been  determined  to  be  approximately  %  degree 
and  vertical  resolution  Just  under  1  degree  using  the  above 
evaluation  technique.  These  values  are  close  to  the  manufacturer's 
specifications  and  the  theoretical  limits  for  this  eye-movement 
measurement  technique  (scleral  reflection). 

i.  Calibration:  Calibration  must  be  the  first  step  in  any 
experiment  in  which  eye-movement  data  are  collected  so  that  the 
calibration  data  can  serve  as  a  reference  to  which  the  experimental 
data  can  be  scaled.  Our  present  calibration  procedure  is  as 
follows: 

(1)  The  subject  is  directed  to  fixate  on  the  appropriate 
points  of  a  square  9-point  matrix  (3  rows  x  3  columns)  on 
the  rear  projection  screen  or  the  storage  scope  while  the 
experimenter  adjusts  the  sensors  so  that  symmetry  of  voltage 
readings  (monitored  on  another  oscilloscope)  around  the  center 
point  is  obtained  for  both  vertical  and  horizontal  movements. 
The  horizontal  and  vertical  extremes  of  this  matrix  subtend 
an  angle  of  20  degrees  at  the  subject's  eye.  A  17-point 
calibration  matrix  of  two  concentric  circles  subtending 
angles  of  10  and  20  degrees  at  the  eye  is  also  used.  The 
calibration  matrices  are  shown  in  Figures  V.3  and  V. 5.  Eye- 
movement  data  obtained  by  directed  fixations  on  the  points 
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of  these  matrices  are  shown  in  Figures  V.4  and  V.6, 
respectively. 

(2)  When  the  experimenter  obtains  what  appears  to  be  the 
best  possible  adjustment,  he  checks  this  by  running  a  cali¬ 
bration  program.  This  calibration  program  samples  the 
horizontal  and  vertical  EMM  voltages  as  the  subject  16 
directed  to  fixate  on  the  individual  points  of  the  calibra¬ 
tion  matrix.  The  average  voltage  for  each  point  is  deter¬ 
mined  and  stored. 

(3)  Various  calculations  are  done  on  these  data  and  results 
printed  so  the  experimenter  can  determine  how  good  the 
sensor  adjustment  is.  The  calculations  include  horizontal 
and  vertical  crosstalk,  zero  shift,  horizontal  and  vertical 
full-scale  values  (indicative  of  symmetry),  and  variations 
of  the  position  voltages  about  the  average  values. 

(4)  If  these  results  are  unsatisfactory,  the  sensors  are  re¬ 
adjusted  appropriately;  and  the  calibration  program  is  re- 
ruii.  This  is  repeated  until  the  experimenter  is  satisfied 
that  he  has  obtained  the  best  possible  calibration. 

(5)  The  desired  experiment  is  performed,  with  the  calibra¬ 
tion  being  checked  at  various  points  of  the  experiment. 

Crasstalk  and  zero  shift  are  the  most  significant  problems 
encountered  in  the  eye-movenent  measurement <3  because  they  result 
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in  inconsistent  data  and  complicate  data  analysis.  Horizontal 
crosstalk  occurs  when  a  non-zero  horizontal  sensor  voltage 
difference  is  produced  for  what  is  a  purely  horizontal  eye-move¬ 
ment.  The  effect  of  this  crosstalk  is  to  distort  the  mapping 
from  the  two-dimensional  eye-movement  cpace  to  the  two-dimensional 
voltage  space  so  that  this  mapping  is  not  a  linear  one-to-one 
transformation.  Thus,  for  a  bad  sensor  adjustment,  a  given  voltage 
reading  may  represent  several  eye  fixations  separated  by  several 
degrees  of  eye-rotation.  Clearly,  we  desire  to  reduce  this 
distortion  as  much  as  possible. 

Vertical  crosstalk  is  negligible,  but  horizontal  crosstalk 
can  be  severe  for  different  subjects  and  on  different  days  for 
the  same  subject,  even  with  optimal  sensor  position.  This 
crosstalk  is  due  tc  the  interference  of  the  upper  and  lower  lids 
with  the  light  received  by  the  horizontal  sensors  as  the  eye  makes 
vertical  movements.  Because  of  the  uneven  curvature  of  the  eye¬ 
lids,  interference  with  each  of  the  two  sensors  is  asymmetrical, 
resulting  in  a  "noise"  voltage  difference  between  the  sensor 
outputs.  However,  since  we  know  the  cause  of  the  distortion, 
we  can  adjust  the  sensors  efficiently  and  quickly  so  as  to 
minimize  the  error.  To  further  circumvent  this  problem,  we  can 
use  the  calibration  results  to  determine  a  linearization  function 
for  the  distorted  regions  and  modify  the  experimental  data  for  this 
region  in  accordance  with  this  function.  Another  approach  is  to 
constrain  the  displays  presented  to  the  subject  so  as  to  avoid 
high-distortion  areas. 
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The  regions  with  the  greatest  amount  of  distortion  include 
those  downward  and  to  the  right  and  upward  and  to  the  left  in  the 
subject's  field  of  vision,  when  eye-movements  of  greater  than  5 
degrees  from  the  center  of  the  display  must  be  i.  ide.  Movements 
within  a  circle  having  its  center  in  the  center  of  the  subject's 
field  of  vision  and  subtending  10  degrees  with  the  eyes  generally 
yield  a  very  good  one-to-one  transformation  of  eye-movement 
position  to  sensor  voltage.  Examples  of  disto’xion  in  the  data 
can  be  seen  in  Figures  V.  4  and  V. 6. 

Zero  shift  is  another  problem,  since  the  output  voltages  of 
the  EMM  are  nulled  (zero  volts)  while  the  subject  fixates  on  the 
center  of  the  display.  Movements  away  from  this  center  point 
create  plus  or  minus  voltages  relative  to  this  null.  Head  movement 
shifts  the  voltage  output  to  a  non-zero  value  for  a  fixation  on 
the  center  point.  Symmetry  of  left-right,  up-down  voltages  is 
maintained  with  respect  to  the  new  offset  "zero",  but  we  want  to 
reduce  this  offset  drift  as  much  as  possible  to  simplify  data 
analysis.  This  is  done  principally  by  constraining  the  subject's 
head  in  the  headrest.  We  may  have  to  resort  to  using  a  bite  bar 
later  in  some  experiments  to  minimize  errors  resulting  from  head 
movements. 

b.  Data  Analysis 

We  have  developed  what  is  essentially  a  more  automatic  version  of 
Noton's  [6]  scanpath  detection  procedure.  Analysis  is  performed  on 
data  digitized  and  stored  on  magnetic  tape.  A  very  flexible  program 


has  been  written  for  accessing  these  data,  performing  calculations  on 
it,  plotting  it  on  the  CRT  or  incremental  plotter,  etc.  This  program 
allows  one  to  access  data  continuously,  stop  and  restart  access  it  any 
time  (enables  one  to  view  plots  on  the  CRT  in  sections,  for  example), 
access  a  single  point,  or  access  any  number  of  sequential  points  in  a 
record  or  file. 

We  currently  have  different  fixation  recognition  and  labeling 
programs.  These  were  written  for  use  in  the  experiments  described 
above  to  facilitate  plotting  and  numbering  sequential  fixations.  The 
recognition  algorithms  are  based  on  the  simple  criterion  that  some 
number  of  sequential  sample  points  be  clustered  within  a  certain  "area" 
(X-Y  voltage  region).  The  number  of  points  is  determined  by  the  time 
interval  between  samples,  since  a  fixation  is  defined  for  ot'r  purposes 
to  be  greater  than  or  equal  to  200  milliseconds.  This  value  is  chosen 
for  two  reasons.  First,  200  ms  is  generally  acknowledged  in  the 
literature  to  be  approximately  the  minimum  duration  of  fixation  (often 
referred  to  as  the  inter-saccadic  interval).  Secondly,  by  looking  at 
our  own  plots  of  eye-movement  data,  we  can  count  the  number  of  sample 
points  in  clusters  which  "look  like"  fixations;  from  this  and  the 
sampling  rate,  we  calculate  the  minimum  fixation  duration  to  be 
approximately  200  milliseconds. 

This  is  a  very  rough  and  somewhat  heuristic  approach  to  defining 
a  fixation,  or  more  importantly,  the  boundary  (in  time)  between  a 
fixation  and  a  saccade.  Tnough  adequate  for  the  above  experiments ,  this 
definition  and  the-  algorithms  implementing  it  are  not  very  useful  for 
the  types  of  experiments  we  hope  to  be  performing  in  the  near  future. 
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Therefore,  the  present  effort  involves  more  accurate  determination  of 
the  fixation  -  saccade  boundary.  This  is  important  because  the  brain 
accepts  visual  data  from  the  retina  only  during  eye  fixation,  not  during 
saccades.  We  should  like  to  predict  the  onset  direction  and  extent 
of  the  saccade  at  the  earliest  possible  moment  and  to  use  various 
parameters  of  the  saccade  (such  as  velocity  and  acceleration)  for  pre¬ 
dicting  the  location  of  the  next  fixation. 

There  are  indications  from  previous  work  by  Yarbus  [9]  that 
certain  saccade  and  fixation  parameters  vary  characteristically  during 
the  time  course  of  a  saccade  or  fixation.  Specifically,  Yarbus  noted 
the  magnitude  of  the  velocity  and  acceleration  of  saccades  very  nearly 
approximated  a  sinusoidal  function  with  respect  to  time.  While  these 
results  may  be  partially  attributable  to  artifacts  in  Yarbus'  measure¬ 
ment  techniques,  the  suggestion  that  velocity  and  acceleration  follow 
a  time  course  having  a  maximum  value  with  perhaps  some  symmetry  about 
this  maximum  it  reasonable  from  a  physiological  point  of  view  of  the 
saccadic  meachanism.  If  we  can  find  a  similar  characteristic  pattern 
of  velocity  and  acceleration,  we  can  use  these  characteristics  to 
predict  not  only  the  onset  of  a  saccade,  but  also  its  direction  and 
terminating  fixation  position.  We  are  looking  for  a  consistent  pattern 
of  parameters. 

Initial  work  on  this  problem  is  being  done  off-line  to  determine 
which  characteristic  patterns  of  what  parameters  exist.  The  next 
step  will  be  to  extract  the  parameters  providing  the  most  important 
information  and  implement  their  analysis  in  a  real-time  environment. 

In  summary, we  have  been  developing  techniques  for  collecting  and 


analyzing  eye-movement  data  for  the  purpose  of  discovering  spatial  cues 
that  can  be  used  as  keys  for  memory  retrieval.  Having  started  with 
off-line  analytical  procedures,  we  are  now  designing  algorithms  for 
real-time  techniques  for  the  extraction  of  the  critical  eye-movement 
parameters  and  using  this  information  in  feedback  control  schemes  to 
influence  impression  and  recall  of  visual  images. 

2.  A  Model  for  Eye-i'ovement  Tracking 

In  order  to  feedback  the  spatial  cues,  it  is  necessary  to  have  an 
estimator  and  predictor  filter  for  the  eye-movement  process.  Such  a 
filter  is  used  in  tracking  the  eye  movement  and  eye  fixation.  This 

information  is  used  to  generate  visual  stimuli  at  appropriate  spots  in 
the  visual  field. 

For  estimation  and  prediction,  it  is  useful  to  have  a  model  of 
the  eye-movement  process.  At  present,  some  models  have  appeared  in 
the  literature,  but  most  of  these  are  concerned  with  eye-tracking 
movements  [l,  2];  i.e. ,  movements  of  the  eye  when  following  a  moving 
target,  and  eye-movement  control  [3,  4].  However,  for  the  study  of 
the  interaction  between  the  memory  and  eye  movements,  we  need  a  nodel 
for  viewing  stationary  patterns.  Recently,  a  few  such  models  have  been 
proposed  [5,  6,  7,  8].  These  have  not,  however,  been  used  for  the 
purpose  of  prediction.  We  shall  develop  a  model  which  can  be  used  for 

this  purpose.  We  shall  first  give  a  brief  account  of  the  existing 
theories. 

A  theory  of  pattern  perception  has  been  propounded  by  Noton  and 
Stark  [5,  6].  This  theory  is  concerned  with  explaining  eye  movements 


during  learning,  subsequent  recognition,  and  recognition  under  un¬ 
favorable  real  world  conditions  (e.g. ,  distortion,  noise,  etc.)  of 
visual  patterns.  The  assumption  is  made  that  the  learning  of  a  pattern 
involves  constructing  an  internal  representation  of  the  pattern  in 
memory  while  recognition  involves  a  search  for  an  internal  representa¬ 
tion  which  matches  the  given  pattern.  Noton  and  Stark  proposed  a 
particular  form  of  the  internal  representation  and  of  the  process  of 
finding  and  matching.  They  observed  that  for  a  particular  subject 
viewing  a  particular  pattern,  there  exists  a  fixed  sequence  of  fixation 
points  which  occurs  for  a  large  fraction  of  the  inital  viewing  time. 

They  call  this  sequence  a  scanpath.  The  scanpath  is  different  for 
different  subjects  and  for  different  patterns  (although  for  a  particu¬ 
lar  pattern,  the  "features"  are  the  same  for  most  subjects).  At  the 
time  of  the  initial  viewing,  about  35$  of  the  time  was  spent  in 
repeating  the  scanpath  with  irregular  fixations  during  the  rest  of  the 
time.  This  scanpath  was  found  to  be  repeated  in  65$  of  later  viewings 
of  the  same  pattern. 

Based  on  these  observations,  Noton  and  Stark  proposed  that  the 
internal  representation  of  an  object  in  memory  is  primarily  serial  in 
naturr*.  It  consists  of  a  network  of  memory  traces,  termed  a  "feature 
network",  which  records  the  features  of  the  pattern  and  the  movements 
required  to  go  from  feature  to  feature  in  the  pattern.  During  learning, 
this  representation  is  formed.  During  recognition,  the  features  of 
the  given  pattern  are  matched  sequentially,  with  the  internal  represen¬ 
tation  directing  attention  to  the  successive  features  of  the  patterns 
and  thus  causing  the  eye-movements.  Such  a  model  would  explain  the 


occurrence  of  scanpaths,  which  are  supposedly  due  to  the  force  of  habit 
in  perception  so  that  each  person  develops  his  own  characteristic  path 
for  a  pattern. 

This  is  a  very  simple  model  which  is  not  very  useful  for  prediction. 
It  can  be  extended  to  a  statistical  model,  where  the  scanpath  is  the 
most  probable  direction  of  eye  movement,  but  there  is  a  probability 
assignment  for  the  other  parts  of  the  feature  network  also.  This  can 
be  used  for  prediction  using  standard  statistical  methods.  It  is  an 
empirical  model,  for  which  the  scanpath  has  to  be  determined  first. 

The  above  model  does  not  give  any  quantitative  way  of  determining 
the  scanpath  in  advance,  but  this  difficulty  can  be  lemedied  by  the 
model  proposed  by  F.  J.  Prokoski  [7].  This  model  is  a  non-linear 
spatio-temporal  control  system  with  the  pattern  as  the  input  and  a 
series  of  vectors  representing  the  eye  movements  as  the  output.  Tem¬ 
poral  and  spatial  sampling  is  assumed  so  that  the  system  is  a  discrete 
time  control  system.  The  image  presented  to  the  eyes  is  first  mapped 
into  the  receptor  response.  The  mapping  function  is  the  relative 
sensitivity  of  the  receptors  as  a  function  of  the  gaze  angle.  Now  a 
mapping  is  made  to  a  short-term  memory.  It  is  assumed  that  the  short¬ 
term  memory  decays  exponentially  with  time  and  the  receptor  response 
is  added  on  directly  to  the  contents  of  this  memory.  The  rate  of 
decay  is  one  of  the  parameters  of  the  model  and  depends  on  the  subject. 

It  is  assumed  that  the  visual  cortex  performs  an  absolute  value 
difference  cross-correlation  between  the  contents  of  the  short  term 
memory  and  the  receptors  response  at  any  given  time.  This  cross¬ 
correlation  is  restricted  to  the  foveal  portion  of  the  receptor  array. 
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The  resulting  function  determines  where  the  eyes  will  be  moved  at  the 
next  sampling  instant.  The  eyes  are  moved  if  the  current  intensity 
sum  over  the  foveal  area  falls  below  a  certain  threshold,  determined 
by  the  short-term  memory  and  a  parameter  representing  the  psychological 
set  of  the  subject.  In  that  case,  the  eyes  are  moved  by  the  control 
vector  determined  by  the  minimum  and  the  origin  of  the  output  of  the 
cross-correlation. 

The  assumptions  made  in  this  model  are  to  some  extent  supported 
by  physiological  and  psychological  data.  The  model  has  been  used  for 
prediction  of  teaching  movements.  With  a  few  extension.*  and  modifica¬ 
tions,  it  can  be  used  for  our  purpose.  For  the  case  of  recognition, 
long  term  memory  has  to  be  incorporated  in  the  model.  Also,  the  model 
at  present  is  deterministic  and  produces  a  specific  output  vector. 

We  can  change  it  to  a  stochastic  control  system  where  the  output  vector 
is  a  random  function  of  the  cross-correlation  array.  This  will  be 
helpful  in  determining  the  accuracy  of  the  predictive  filter  in  a 
quantitative  sense.  The  model  can  be  useful  at  least  for  the  initial 
production  of  the  scanpath. 

Another  model  to  be  considered  is  a  stochastic  model  [8]  which, 
as  given,  applies  only  to  eye  movements  during  fixation  and  not  for 
saccadic  movements  between  fixations.  Within  a  fixation,  these  are 
"local  areas"  of  fixat<on  and  transitions  from  one  local  area  to 
another.  Theoretical  transforms  of  the  probability  density  function 
for  local  area  motion  and  the  moments  of  the  number  of  transitions  in 
a  given  time  T  are  obtained.  This  model  can  probably  be  extended  to 
cover  transition  from  fixation  to  fixation.  By  looking  at  the  fixation 


data,  we  should  be  able  to  predict  when  the  fixation  ends. 

The  above  summarizes  the  various  types  of  models  we  will  be 
considering  for  our  use.  Extensions  and  modifications  to  any  one  or 
more  of  these  can  make  them  suitable  for  our  purpose  of  estimation  and 
prediction. 

Now  we  will  briefly  describe  the  v/ork  on  eye  movement  data  collec 
tion  and  analysis  which  has  been  accomplished.  The  instrument  used 
for  measuring  eye  movements  uses  the  scleral  reflection  technique 
modified  to  measure  vertical  and  horizontal  movements.  The  operation 
of  the  instrument,  the  data  gathering  programs  and  the  calibration 
methods  have  been  describe  elsewhere  in  this  report.  Our  initial 
efforts  have  been  directed  toward  verifying  some  of  Noton  and  Stark's 
experiments  on  scanpaths.  Noton  and  Stark  slowed  down  the  eye-move¬ 
ment  data,  plotted  it  and  marked  the  fixation  points  by  hand.  An 
algorithm  for  determining  the  fixation  points  is  implemented  on  the 
PDP-15.  A  brief  description  follows. 

Raw  data  are  read  off  the  magnetic  tape.  Calibration  data, 
obtained  in  the  same  experiment,  are  used  in  processing  these  raw 
data.  The  calibration  assumes  linearity  although  different  reference 
values  are  used  for  each  of  the  four  quadrants.  This  is  done  to  take 
care  of  the  crosstalk  and  the  asymmetry  in  the  calibration  data. 

After  calibration,  the  fixation  program  has  essentially  two  parts 
a)  determining  when  the  fixation  mode  has  been  entered,  and  b)  once  a 
fixation  is  decided  upon,  determining  when  it  ends.  Both  use  certain 
parameters  which  are  largely  empirical  in  nature.  For  determining  a 
fixation,  the  program  checks  out  whether,  of  N  points,  M  lie  within  a 
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certain  distance  D  of  the  mean  value.  The  values  of  N,  M,  and  D  have 
been  set  from  reported  studies  [9]  on  duration  and  spread  of  fixations 
and  from  our  own  observations.  To  determine  the  end  of  the  fixation, 
the  program  checks  if  a  certain  number  of  consecutive  points  fall  out¬ 
side  of  a  prescribed  mean.  Again,  the  parameter-  have  been  set  as  above. 

The  fixation  points  are  plotted  and  numberei  and  their  locations 
printed.  ThU3  we  can  automatically  determine  the  sequence  of  fixations 
and  observe  the  scarpatli. 

The  performance  of  this  program  has  been  tested  by  comparing  it 
with  the  determination  of  fixations  by  looking  at  the  plotted  data  and 
marking  the  apparent  fixation  points.  It  has  been  found  to  be  very 
satisfactory.  Some  examples  are  shown  in  Figures  V.  7,  V.8,  V.9,  and 
V.  10. 

In  addition  to  this,  we  have  programs  for  plotting  data,  both  on 
the  CRT  and  the  CalComp  plotter,  for  analysis  of  fixation  points 
(standard  deviation,  frequency  histograms  of  length  of  movements, 
distribution  of  angle  of  movements,  etc.),  for  separate  plotting  of 
the  horizontal  and  vertical  movements  and  their  analysis,  etc.  Other 
programs  are  being  written  to  facilitate  the  task  of  data  analysis 
and  model  building. 


59 


Figu -e  V.8  The  sequence  of  fixations  in 
the  above  as  determined  by  the  computer  program 
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Figure  V.9  Plot  of  eye  novement 
for  another  pattern  (20  secs. ) 


Figure  V. IT  The  sequence  of  fixation  in  the  above 


61 


REFEFENCES 


[1]  L.  Stark,  G.  Vossius  and  L.  R.  Young,  "Predictive  Control  of  Eye 
Tracking  Movements  ,  IRE  Trans,  on  Huiu't  Factors  in  Electronics, 
HFE-3,  52-57,  September  1962. 

[2]  L.  R.  Young  and  L.  Stark,  "Variable  Feedback  Experiments  Testing 
a  Sampled  Data  Model  for  Eye  Tracking  Movements",  IEEE  Trans,  of 
the  Professional  Tech.  Group  on  Human  Factors  in  Electronics, 

HFE-4 ,  38-51,  1963. 

[3]  G.  Vossius,  "The  Control  of  Eye-Movement",  Proc.  of  the  1972 
Int'l  Conference  on  Cybernetics  and  Society,  27-31. 

[4]  D.  A.  Robinson,  "Progress  in  Models  of  Eye  Movement  Control", 

Proc.  of  the  1972  Int'l  Conference  on  Cybernetics  and  Society, 
14-24. 

[5]  D.  Noton  and  L.  Stark,  "Scanpaths  in  Saccadic  Eye  Movements  While 
Viewing  and  Recognizing  Patterns",  Vision  Research,  Vol.  II, 
929-942,  September  1971. 

[6]  D.  Noton,  "Two  Models  of  the  Brain  with  Applications  to  Artificial 
Intelligence",  Doctoral  dissertation,  University  of  California, 
Berkeley,  California,  1970. 

[7]  F.  J.  Prokoski,  "Modeling  the  Human  in  Visual  Tasks",  Proc.  of  the 
1972  Int'l  Conference  on  Cybernetics  and  Society,  44-54. 

[8]  R.  Vasudevan,  A.  V.  Phatak  and  J.  D.  Smith,  "A  Stochastic  Model 
for  Eye  Movement  During  Fixation  on  a  Stationary  Target",  Technical 
Report  No.  71-23,  Dept,  of  Electrical  Engineering,  University  of 
Southern  California,  Los  Angeles,  California,  May  1971. 

[9]  A.  L.  Yarbus,  Eye  Movements  and  Vision.  New  York,  Plenum  Press. 

1967.  - - 


62 


VI.  CONCLUSION 


The  work  has  been  proceeding  along  the  lines  that  we  proposed. 

We  have  brought  up  the  new  PDP-15  computer  system  to  full  operational 
status.  We  have  investigated  and  implemented  some  real-time  monitoring 
and  prediction  schemes  and  some  modeling  techniques  for  EEG  and  eye- 
movement  variables.  For  instance,  we  have  implemented  a  system  using  a 
discrete  phase-locked-loop  that  makes  possible  real-time  tracking  of 
amplitude,  frequency,  and  phase  variables  in  the  EEG  signals  and  we 
have  implemented  an  algorithm  for  tracking  the  visual  scanpath  and 
automatic  marking  of  the  fixation  points  in  the  scanpaths.  Since  the 
inception  of  this  project,  we  have  had  several  papers  published  and 
some  are  scheduled  to  be  presented/published.  During  the  next  reporting 
period  we  shall  concentrate  on  the  acquisition  of  experimental  data. 

By  using  the  continuously  up-dated  information  concerning  the  eye 
position  and  brain  state  for  adjusting  the  stimulus  parameters,  we 
will  attempt  to  enhance  the  vividness  and  persistence  of  the  desired 
after-images.  To  be  more  specific,  we  plan  to  accomplish  the  following 
tasks  i  n  the  coming  year: 

1)  Develop  real-time  methods  for  relating  eye-movements  to 
visual  targets;  further,  record  the  scanpaths  for  subsequent 
computer-aided  image-enhancement . 

2)  Analyze  the  perceptual  consequences  of  visual  target 
presentation  that  is  contingent  upon  a-phase;  analyze  the  effects 
upon  imagery  of  phase-contingent  blank-field  probing. 


3)  Develop  methods  for  the  estimation  of  eye-pointing  based  on 
the  combination  of  optical  eye-tracking  measurements  and 
electrooculographic  measurements  so  as  to  achieve  a  best  single 
estimator  for  both  eye-open  and  eye-closed  conditions. 

4)  Develop  real-time  strategies  for  the  control  of  visual 
target  impression  and  visual  image  (memory)  persistence  and/or 
recall  ability. 
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APPENDIX  A:  GS-lu--A  GRAPHICS  SOFTWARE  SYSTEM 

1.  Introduction 

GS-15  is  a  generalized  graphics  software  system  for  the  PDP-15 
with  RSX-PLUS.  The  system  provides  general  symbolic  and  numeric 
capabilities  that  can  be  combined  to  perform  any  practical  graphics 
function. 

a.  System  Configuration 

In  order  to  use  GS-15,  a  PDP-15  with  RSX-PLUS  is  required.  How¬ 
ever,  since  a  major  portion  of  the  software  is  coded  in  FORTRAN,  the 
system  could  be  transferred  (albeit  with  effort)  to  another  machine. 
GS-15  is  currently  using  the  FP15  floating  point  hardware  option,  but 
could  more  than  survive  in  a  non-floating  point  environment.  In 
aduition,  our  real-time  laboratory  also  includes  a  VP15A  storage 
scope  and  a  CalComp  incremental  plotter. 

b.  Design  Objectives 

i.  LUN  Orientation:  A  primary  design  consideration  was  to 
create  a  system  that  was  a  logical  unit  (LUN)  rather  than  device 
oriented.  Past  experience  demonstrated  the  disadvantages  of 
device-specific  graphics  software.  Furthermore,  we  wanted  a 
system  that  would  blend  unnoticed  into  RSX. 

ii.  Extensibility:  The  other  main  consideration  was  extensibility. 
Since  our  project's  goal  is  enhancement  of  human  memory  and  not 
systems  software  development,  we  knew  we  would  not  have  time  to 
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create  a  complete  graphics  system.  Instead,  the  decision  was 
made  to  create  the  nucleus  of  such  a  system. 

For  this  reason,  GS-15  does  not  include  large  conglomerate 
user  subroutines  (e.g.,  a  single  subroutine  to  draw  a  complete 
plot  including  labeled  axes,  tick  marks,  etc.),  but  instead  con¬ 
tains  the  essential,  elementary  capabilities  found  in  the  inter¬ 
section  of  currently  available  graphics  systems.  Also  included 
is  the  capability  to  easily  rdd  additional  graphics  functions 
(FORTRAN  subroutines)  as  neec ed  by  the  user.  Furthermore,  as  a 
consequence  of  LUN  oriented  I/O,  all  that  is  required  for  GS-15 
to  utilize  newly  acquired  graphics  hardware  is  the  writing  of 
another  device  handler.  Once  the  handler  is  installed,  previously 
written  graphics  tasks  may  immediately  use  the  new  device. 

2.  Overview 

As  shown  in  Figure  A.l,  GS-15  consists  of  three  major  system  com¬ 
ponents:  the  graphics  pool,  subroutines,  and  device  handlers. 


Figure  A.l  GS-15  Data  Flow 


In  order  to  produce  a  plot,  the  programmsr  must  first  create  a  task 
(program)  containing  calls  to  one  or  mort  GS-15  subroutines.  Next,  the 
appropriate  LUN  should  be  assigned  to  a  graphics  device  and  the  task 
requested. 

When  the  task  CALLS  a  graphics  subroutine,  an  I/O  request  node 
is  queued  to  the  appropriate  logical  unit.  All  necessary  IOTs  are 
executed  as  soon  as  the  device  handler  obtains  control.  When  the 
device  handler  receives  the  last  interrupt  signifying  the  completion 
of  the  particular  graphics  function  requested,  the  caller's  event 
variable  is  set  to  plus  one  (assuming  no  errors  were  detected).  All 
during  this  process,  the  pool  is  referenced  and  updated — always  reflec¬ 
ting  the  current  state  of  the  graphics  job. 

a.  POOL 

The  graphics  POOL  (a  FORTRAN  array)  is  an  integral  part  of  GS-15. 
Parameters  that  are  common  to  many  subroutines  are  collected  together 
in  the  POOL  (see  Table  A.l)  rather  than  being  specified  repetitively  in 
each  subroutine  call.  Typical  elements  of  the  POOL  includ.  the  graphics 
LUN  (needed  by  all  subroutines),  character  height  and  width  (needed  by 
all  symbolic  subroutines),  and  scaling  factors.  In  addition,  certain 
POOL  elements  contain  information  on  the  internal  state  of  GS-15  (e.g.  , 
current  location  of  the  plotter  pen). 

Another  advantage  of  the  POOL  is  maintaining  upward  compatibility 
of  the  GS-15  software.  For  example,  a  user  might  want  a  subroutine 
extended  slightly  (e.g.,  allowing  multiple  character  fonts)  necessi¬ 
tating  the  addition  of  another  parameter.  With  GS-15,  the  new  feature 


could  be  implemented  by  creating  a  new  POOL  element  (font  type)  and 
changing  all  symbolic  routines  to  recognize  this  new  element.  Since 
no  calling  sequences  would  be  changed,  the  modification  would  be  up¬ 
ward  compatible. 


# 

NAME 

DESCRIPTION 

1 

GLUN 

Graphics  LUN 

2 

GFNAME 

Graphics  file  name 

4 

DNSITY 

L'.ne  density 

5 

SPEED 

Speed  factor 

8 

XMIN 

10 

YMIN 

12 

XMAX 

Subject  space 

14 

YMAX 

16 

PXMIN 

18 

PYMIN 

20 

PXMAX 

Object  space 

22 

PYMAX 

24 

XFACTR  ) 

26 

YFACTR  J 

Internal  constants 

28 

PUPD 

Pen  up/down  flag 

29 

CPX  ) 

30 

CPY  / 

Current  pen  position 

33 

STHETA  ) 

35 

CTHETA  /  1 

Symbolic  orientation 

37 

height 

Sumbolic  height 

38 

WIDTH 

Symbolic  width 

1 

Table  A.  1  Graphic;-  POOL  Contents 


b.  Subroutines 

The  second  major  component  in  the  G3-15  system  is  a  set  of 
(FORTRAN/MACRO  callable)  graphics  subroutines  which  provide  the  user 
interface  to  GS-15.  Their  purpose  is  to  make  available  a  set  of 
elementary  graphics  functions  that  the  user  can  combine  into  Targer, 
special  purpose  graphics  routines  which  might,  for  example,  draw  poly- 
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gons  or  three-dimensional  figures.  These  elementary  functions  are 
realized  by  queuing  one  or  more  I/O  request  nodes  to  the  graphics  LUN 
specified  in  the  first  POOL  element.  When  all  such  I/O  requests  have 
been  completed,  the  event  variable  specified  in  the  subroutine  CALL  is 
set  to  indicate  that  the  desired  function  has  been  performed. 

The  last  formal  parameter  in  most  GS-15  subroutines  is  an  event 
variable.  If  an  event  variable  is  not  specified  in  a  graphics  sub¬ 
routine  CALL,  the  subroutine  will  RETURN  to  the  caller  only  after  the 
requested  function  has  been  completed.  On  the  other  hand,  control  is 
returned  immediately  if  an  event  variable  is  present  in  the  subroutine 
CALL.  By  later  checking  the  status  of.  the  event  variable  with  the 
WAITFR  system  directive,  asynchronous  operation  can  be  achieved. 

Table  A. 2  lists  the  subroutines  available  in  the  first  version  of 
GS-15.  It  should  be  kept  in  mind  that  the  action  performed  by  a  sub¬ 
routine  depends  on  the  current  assignment  of  the  graphics  LUN.  For 


NAME 

FUNCTION 

GBEGIN 

Initialization 

GEND 

Termination 

GLINES 

Connect  "n"  points 

GMOVE 

Pen  movement 

GNEXT 

Erase  scope/advance  paper 

GNUMBR 

Draw  formatted  numbers 

IGPX  ) 

IGPY  f 

Translate  subject  to  object  space 

GSCALE 

Define  subject  and  object  spaces 

GSET 

Modify  POOL  elements 

GSYMBL 

Draw  single  character 

GTEXT 

Draw  groups  of  characters 

Table  A.  2  GS-15  Subroutines 


example,  GNEXT  erases  the  scope  if  GLUN  is  assigned  to  the  storage 
CRT,  or  advances  the  plotter  paper  beyond  the  furthest  point  drawn  in 
the  positive  "X"  direction  if  assigned  to  the  plotter. 

c.  Device  Handlers 

Graphics  device  handlers  enable  GS-15  to  control  the  graphics 
hardware.  They  are  the  recipients  of  the  I/O  request  nodes  queued  by 
the  graphics  subroutines.  IOTs  are  issued  by  the  handlers  based  on 
graphics  commands  contained  within  the  request  nodes.  In  order  to  be 
effective  in  the  RSX  environment,  the  handlers  recognize  most  system 
I/O  function  codes  (ABORT,  ATTACH,  etc.)  in  addition  to  the  non-standard 
codes  needed  by  the  GS-15  system. 

Currently,  two  graphics  device  handlers  exist:  TV,  which  controls 
a  VP15A  storage  scope,  and  PL,  which  controls  a  CalComp  incremental 
plotter.  Both  were  written  in  MACRO-15  and  occupy  less  than  2000 

8 

words  of  memory.  Since  TV  and  PL  are  very  similar,  only  one  source 
file  (TVPL)  is  maintained  with  conditional  assemblies  producing  the 
appropriate  binary  files. 

A  new  handler  (GF)  will  be  added  shortly.  Assigning  GLUN  to  GF 
wil1  cause  a  graphics  file  (whose  name  is  in  the  second  POOL  element) 
to  be  created  on  disk  or  dectape  depending  on  the  current  assignment  of 
LUN  17  (normally  assigned  to  disk).  Once  the  file  has  been  created, 
it  may  be  processed  by  a  low  priority  background  task  and/or  saved 
for  future  use.  This  feature  will  provide  the  fastest  mode  of 
operation  for  GS-15.  With  it,  "snapshots"  of  real-time  processes  may 
be  stored  on  the  disk  and  later  selectively  viewed. 
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3. 


Usage 


a.  Task  Structure 

As  mentioned  previously,  a  graphics  task  contains  calls  to  various 
GS-15  subroutines.  The  general  structure  of  a  graphics  task  is  shown 
in  Figure  A.  2 

FORTRAN  declarations 
I - 

CALL  GBEGIN 

FORTRAN  statements 
and 

CALLS  to  GS-15 

CALL  GEND 

STOP 

END 

Figure  A.  2  Typical  Graphics  Task 

GBEGIN  must  be  the  first  GS-15  subroutine  calleu,  and  may  be  called 
as  many  times  as  desired.  Its  purpose  is  to  initialize  both  the  POOL 
and  the  currently  assigned  graphics  device  handler.  Although  not  a 
requirt ment  of  GS-15,  GEND  should  be  called  before  the  task  terminates 
in  order  to  rllow  GS-15  to  perform  certain  "cleanup"  procedures. 

b.  Subject/Object  Spaces 

A  common  problem  in  computer  installations  with  graphics  facilities 
is  adapting  "real  data"  to  the  addressing  schemes  used  by  graphics 
hardware  devices.  For  example,  a  scientist  might  want  to  draw  the 
graph  of  an  electrical  waveform  on  a  CRT.  The  voltages  corresponding 
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to  the  actual  waveform  would  take  on  real  values  between,  say,  -10 
volts  and  +10  volts  (e.g.,  1.625,  -8.301,  0.108,  etc.).  The  CRT, 
however,  requires  a  pair  of  Integers  to  address  a  particular  point  on 
Its  display  surface  (e.g.,  the  Tektronix  611  storage  scope  contains 
a  1024  x  1024  grid  of  such  points).  The  problem  is  to  create  a  simple 
mechanism  in  the  graphics  system  to  convert  real  valuer  data  (which, 
in  absolute  value,  could  approach  10?5)  into  integer  data  (in  the 

range  0  to  1023).  GS-15  solved  this  problem  by  utilizing  subject  and 
object  spaces. 


i.  Object  Space:  As  shown  in  Figure  A. 3,  the  object  space  is  a 
parameterized  subset  of  the  first  quadrant  of  a  discrete 
euclidean  two-space.  Note  that  a  general  point  (IX,  IY) ,  in  the 


+X 


PXMIN  S  IXS  PXMAX 
PYMIN  S  IY  S  PYMAX 


IX,  IYC{0,  1,  2,  ..j 
Figure  A.  3  Object  Space 


object  space,  is  defined  by  an  ordered  pair  of  non-negative 
— -egers<  Furthermore,  the  object  space  may  be  redefined  by 
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changing  (see  G SCALE  below)  the  values  of  PXMIN,  PYMIN ,  PXMAX, 
and  PYMAX. 

The  purpose  of  the  object  space  is  to  define  where  on  the 
graphics  device  the  user  wants  plotting  to  be  done.  For  example, 
to  utilize  the  full  face  of  a  Tektronix  611  CRT,  PXMIN  and  PYMIN 
should  be  set  equal  to  0  (zero)  and  PXMAX  and  PYMAX  should  be  set 
equal  to  1023.  Other  uses  of  variable  object  spaces  are  described 
below  (see  next  section). 

The  discrete  two-space  of  which  the  object  space  is  a  subset 
is  identical  to  the  address  space  present  in  all  incremental  (i.e. , 
digital)  graphics  devices.  In  other  words,  most  devices  allow  the 
user  to  either  connect  (plotter)  or  illuminate  (CRT)  any  of  a 
number  of  points  in  an  imaginary  grid  on  the  device.  T\it  portion 
of  this  grid  that  is  of  interest  to  the  programmer  is  precisely 
the  object  space.  GS-15  assumes  that  the  object  space  is  wholely 
contained  within  the  usable  address  space  of  the  graphics  device. 

Li.  Subject  Space:  As  mentioned  earlier,  actual  data  does  not 
lend  itself  to  direct  expression  in  the  object  space.  For  this 
reason,  the  subject  space  was  created.  The  subject  space  is  a 
subset  of  the  normal  (non-discrete)  euclidean  plane  (see  Figure  A. 4). 
Unlike  object  space  coordinates,  points  in  the  subject  space  are 
specified  by  pairs  of  real  numbers.  This  subject  space  may  also 
be  defined  by  the  user  through  use  of  the  GSCALE  subroutine. 

Normally,  the  subject  space  is  defined  such  that  it  encloses 
all  the  data  points  that  are  to  be  displayed.  For  example, 
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assume  a  plot  of  SIN(0)  is  desired  for  -  — —  s  0  S  n,  In  this 

2 

case,  the  subject  space  could  be  defined  as 

JOHN  =  -  JL-,  XMAX  =  it  ,  YMIN  =  -1,  and  YMAX  =  1. 


-lo75  £  XMIN  £  X  £  XMAX  £  io75 
-1075  £  YMIN  £  Y  £  YMAX  £  10?5 

*.  yc(R 


Figure  A.  4  Subject  Space 

iii.  Subject/Object  Mapping:  If  by  chance  the  data  to  be  plotted 
is  integer-valued  and  within  the  object  space  (e.g.  ,  a  graph  of 
time  vs.  digitized  analog  signals),  the  plot  may  be  produced  with¬ 
out  the  need  of  the  subject  space.  On  the  other  hand,  suppose 
that  the  data  points  to  be  plotted  are  not  contained  in  the 
object  space  (a  more  likely  situation). 

GS-15  contains  two  integer-valued  FORTRAN  functions  (see 
IGPX  and  IGPY  below)  that  define  a  mapping  or  projection  of 
of  the  subject  space  onto  the  object  space  (i.e.  ,  onto  a  portion 
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of  the  graphics  device).  This  mapping  performs  the  necessary 


scaling  and  translation  from  real-valued  data  to  the  integer 


valued  points  in  the  object  space  (see  Figure  A.  5) 


SUBJECT 
^ SPACE 


OBJECT 

SPACE' 


(XMIN.YMAX) 


(PXMIN,  PYMAX) 


(PXMIN.PYMII 


(PX  MAX,  PYMAX) 


(XMAX.YMAX) 


(PXMAX,  PYMIN) 


(XMAX,  YMIN) 


Figure  A. 5  Subject/Object  Space  Projection 


Currently,  most  GS-15  subroutines  are  oriented  towards  the 


object  space.  One  exception  is  GLINES  which  will  connect  together 


N  points  in  the  subject  space.  In  the  future,  other  subject 


space  oriented  routines  will  be  written.  It  should  be  pointed 


out  that  subroutines  that  expect  object  space  coordinates  can 


work  with  subject  space  coordinates  by  using  IGPX  and  IGPY. 

The  use  of  the  subject/object  space  mapping  has  advantages 
that  may  not  be  apparent.  One  very  useful  device  is  that  of 
linearly  expanding  or  contracting  a  picture  by  changing  the 
subject  space  definition.  In  addition  to  linear  translations, 
rotations  about  one  or  both  axes  can  be  performed.  For  example, 
subject  space  data  may  be  rotated  about  the  "x"  axis  by  inter¬ 
changing  the  values  of  YMIN  and  YMAX. 

c.  Symbolic  and  Numeric  Modes 

GS-15,  like  most  graphic  systems,  has  two  modes  of  operation.  The 
first  and  most  frequently  used  mode  allows  the  user  to  draw  lines, 
curves,  etc.  This  mode  (numeric  or  line  drawing)  is  used  to  plot  data 
and  draw  figures  (one  user  has  drawn  a  helicopter  in  flight).  In 
fact,  numeric  mode  subroutines  (GMOVE ,  GLINES)  allow  the  programmer  to 
have  complete  control  of  the  hardware.  For  example,  GMOVE  (when  GLUN 
is  assigned  to  PL)  is  able  to  raise/lower  the  plotter  pen  and  move  the 
pen  to  any  position  that  the  hardware  can  reach.  Subroutines:  that 
expect  subject  space  parameters  usually  operate  in  numeric  mode. 

In  symbolic  mode  strings  of  characters  may  be  drawn  whose  height, 
width  and  orientation  depend  on  values  in  the  POOL.  Because  textual 
annotations  are  usually  placed  at  specific  locations  on  the  graphic 
device  (e.g. ,  3"  from  bottom  of  CRT  and  centered),  symbolic  subroutines 
(GSYMBL,  GTEXT)  are  oriented  towards  the  object  space. 

d.  POOL 

As  mentioned  earlier,  the  POOL  is  t.  vector  of  parameters  common 
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to  many  subroutines.  A  bri  f  explanation  of  the  POOL  elements  follows. 

i.  GLUN:  This  element  contains  the  logical  unit  number  (LUN)  on 
which  GS-15  I/O  request  nodes  are  queued  (see  GQUEUE  subroutine). 

The  default  value  of  GLUN,  23,  corresponds  to  the  system  graphics 
LUN. 

ii.  GFNAME :  When  the  graphics  file  device  handler  (GF)  is 
operational,  I/O  request  modes  may  be  written  into  a  file  whose 
name  is  in  GFNAME  by  assigning  GLUN  to  GF.  This  file  will  be 

written  on  the  device  corresponding  to  LUN  17  (either  DECtape  or 
disk). 

iii.  DNSITY:  DNSITY  defines  the  number  of  points  per  inch  in 
lines  drawn  by  GS-ll  The  CRT,  for  example,  need  not  illuminate 
every  point  in  order  to  draw  well-defined  lines.  Decreasing  the 
DNSITY  can,  therefore,  speed  up  execution  without  creating  a 
noticeable  loss  in  quality. 

iv.  SPEED:  This  element  allows  the  user  to  conveniently  cause  an 
additional  delay  between  the  connecting/illuminating  of  points. 
SPEED  =  0  will  cause  no  dealy.  SPEED  /  0  will  cause  a  delay  of 
SPEED  number  of  clock  ticks  (on  our  machine  1  tick  =  1  msec) 
between  points. 

This  option  has  two  uses.  First,  for  many  applications,  the 
CRT  will  draw  lines  too  quickly.  Setting  SPFED  to  non-zero  allows 
the  displaying  to  be  done  as  slow  as  the  user  wishes.  A  second 
advantage  appears  when  RSX  is  being  used  in  a  multi-programming 
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mode  Ci.e.  ,  more  than  one  task  executing  simultaneously).  Since 
the  delay  is  implemented  using  the  MARK  and  WAITFR  system 
directives,  the  processor  may  be  used  by  other  tasks  during  the 
delay  period.  The  processor  may  not  be  used  by  another  task  when 

running  at  default  SPEED  (0)  since  the  CRT  graphics  functions  are 
not  interrupt  driven. 

v.  XMIN-PYMAX:  These  eight  values  define  the  current  subject 

and  object  spaces.  They  may  be  changed  through  use  of  the  GSCALE 
subroutine. 

vi.  XFACTR/YFACTR:  In  order  to  speed  up  the  calculation  of  the 
mapping  function  (see  IGPX  and  IGPY  subroutines)  between  the  sub¬ 
ject  and  object  spaces,  these  internal  constants  are  computed  and 
kept  in  the  POOL.  They  are  changed  whenever  GSCALE  is  called. 

Vii.  PUPD:  PUPD  keeps  track  of  whether  the  pen  is  currently  up 
(=1)  or  down  (=0). 

CPX/CPY:  These  elements  contain  the  current  location  of  the 
pen  within  the  object  space. 

STHETA/CTHETA :  Theta  is  defined  as  the  symbolic  orientation 
angle  (in  degrees).  STHETA  and  CTHETA  contain  respectively  the 
SIN  and  COS  of  theta.  Theta  is  considered  to  be  element  33  in  the 
POOL  and  may  be  set  by  using  GSET  (see  below). 

*'  HEIGHT/WIDTH:  These  elements  define  (in  inches)  the  size  of 
characters  drawn  by  GSYMBL  and  GTEXT. 
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e.  Subroutines 

Following  is  a  description  of  all  user  subroutines  available  in 
GS-15.  Other  subroutines  exist  (see  section  on  implementation)  that 
are  used  internally  by  GS-15. 


GBEGIN 


Parameters:  None 


Function: 


Initialize  the  POOL  to  default  values  (as  shown  in  Table 


A. 3).  Initialize  the  device  handler  currently  assigned  to 
GLUN.  Must  be  first  GS-15  subroutine  called.  May  be 
called  as  many  times  as  necessary  without  intervening 
GEND. 


POOL 

Element 

Default 

Value 

GLUN 

23 

GFNAME 

"gfile" 

DNSITY 

100 

SPEED 

0 

XMIN 

-100.0 

YMIN 

-100.0 

XMAX 

100.0 

YMAX 

100.0 

PXMIN 

0.0 

PYMIN 

0.0 

PXMAX 

1023. 0 

PYMAX 

1023.0 

STHETA 

0.0 

C THETA 

1.0 

HEIGTH 

.233" 

WIDTH 

.166" 

Table  A, 3  POOL  Default  Values 


GEND 


Parameters: 

Function: 


None 

Perform  GS-15  termination  procedures.  No  further  graphics 
operations  are  allowed  until  GBEGIN  is  re-entered. 


GLINES  (I,  N,  X,  Y  [,  IEV] ) 


Parameters : 


Function: 


I  Denotes  initial  point  in  X  and  Y 

N  Number  of  points  to  connect 

X  Real  vector  of  abscissa  values  in  subject  space 

Y  Real  vector  of  ordinate  values  in  subject  space 

Starting  at  (X(I),  Y(I)),  connect  "N"  points  in  the 
subject  space  to  produce  "N-l"  line  segments. 


GMOVE  (IX,  IY,  JNJ[ ,  IEV] ) 


Parameters:  IX  i 
IY/ 


Object  space  lo:ation 


Join/No  join  flag. 


Function: 


Move  the  pen  from  the  current  location  to  (IX,  IY).  if 
JNJ  =  JOIN",  draw  a  line  between  the  two  positions.  If 
JNJ  =  "NOJO",  move  pen  to  (IX,  IY)  without  drav  ing  a  line. 
Both  IX  and  IY  must  be  S  0. 


GNEXT 


Parameters : 
Function: 


None 

If  GLUN  is  assigned  to  TV,  erase  the  CRT. 


If  assigned  to 
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PL,  move  plotter  paper  beyond  furthest  point  drawn  in 


"+X”  direction. 


GNUMBR  (IX,  IY,  FORMAT,  VALUE  [,  IEV]) 

Parameters:  IX) 

IY >  Object  space  location 

FORMAT  Format  specification 

VALUE  Value  to  be  displayed 

Function:  GNUMBR  plots  a  formatted  number  on  the  current  graphics 

device.  As  in  GTEXT,  IX  and  IY  are  the  object  space 
coordinates  of  the  leftmost  character  in  the  numeric 
field.  FORMAT  is  a  <string>  of  from  1  to  5  characters 
that  defines  a  FORTRAN  format  specification  (e.g. ,  ’17', 
' F10. 6'  ,  ' E8. 6' ,  'D16.  9'  ,  ’06’).  Note  that  (IX,  IY) 


specifies  the  position  of  the  leftmost  character  in  the 
field  which  may  or  may  not  be  the  leftmost  non-blank 
character. 

The  number  or  expression  to  be  displayed  is  the 
fourth  parameter.  It  may  be  INTEGER,  REAL,  DOUBLE 
PRECISION,  or  LOGICAL.  It  mey  not  be  DOUBLE  INTEGER.  As 
in  a  WRITE  statemtut,  the  type  of  VALUE  should  match  the 
type  implied  by  the  format  specification. 


IGPX(X) 


Parameters:  X  Abscissa  value  in  subject  space 

Function:  IGPX  is  a  function  that  returns  an  integer  value  equal  to 
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the  object  space  equivalent  of  X.  The  mapping  function 
is  defined  by 


IGPX(X)  =  PXMAX (XMIN-X)  +  PM IN (X-XMAX) 

XMIN  -  XMAX 


IGPY(Y) 

Parameters:  Y  Ordinate  value  in  subject  space. 

Function:  Like  IGPX,  IGPY  returns  the  object  space  equivalent  of  Y 

The  mapping  function  is  defined  by 

IGPY(Y)  =  PYMIN(YMAX~Y)  +  PYMAX (Y-YMIN) 

YMAX  -  YMIN 


G SCALE  (XMIN,  YMIN,  XMAX,  YMAX,  PXMIN,  PYMIN ,  PXMAX,  PYMAX  [,  IEV]) 


Parameter?:  XMIN 
YMIN 
XMAX 
YMAX 


New  subject  space 


Function: 


PXMIN 

PYMIN 

PXMAX 

PYMAX 


New  object  space 


Redefine  subject  and/or  object  spaces.  XMIN, 
must  be  floating  point  quantities. 


PYMAX 


GSET  (I,  VALUE  [,  IEV]) 


Parameters: 


Function: 


I  Pool  index  (see  Table  A.l) 

VALUE  Nev  POOL  element  value 

Set  POOL  element  "i"  equal  to  VALUE.  VALUE  must  be  a 
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floating  point  quantity. 


GSYMBL  (IX,  IY,  ICODE  [,  IEV] ) 


Parameters: 


Function: 


Object  space  location 

ICODE  Symbolic  code  (see  Table  A. 4). 

Draw  the  symbol  corresponding  to  ICODE  at  (IX,  IY).  The 
size  and  orientation  of  the  symbol  will  be  determined 
by  the  POOL  (see  Table  A.l).  Symbols  with  ICODE  <  40g 
will  be  drawn  centered  about  (IX,  IY). 


GTEXT  (IX,  IY,  N,  TEXT  [,  IEV] ) 


Parameters: 


Function: 


Object  space  location 

Number  of  characters 
Characters  to  be  drawn 
Draw  the  first  "n”  characters  of  TEXT  at  (IX,  IY).  TEXT 
may  be  of  three  forms:  a  JORTRAN  <string  constant)  , 
(string  variable),  or  an  (array  element).  Note  that 
TEXT  may  not  be  an  (array  name).  Following  are  examples 
of  the  three  forms: 


IX\ 

IY; 

N 

TEXT 


Call  GTEXT  (0,  100,  4,  ’TEST’,  IEV) 

Call  GTEXT  (512,  512,  2,  CHARS,  IEV) 

Call  GTEXT  (0,  0,  a  7,  BUFFER (3),  IEV) 

The  size  and  orientation  of  the  TEXT  will  be  determined 
by  1  he  POOL  (see  Table  A.l). 
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000 

o 

040 

001 

□ 

041 

0 

■ 

002 

A 

042 

DD 

003 

A 

043 

u 

004 

X 

044 

$ 

005 

+ 

045 

% 

006 

* 

046 

& 

007 

047 

7 

010 

050 

( 

Oil 

051 

) 

012 

052 

X 

013 

053 

+ 

014 

054 

9 

015 

055 

— 

016 

056 

■ 

017 

057 

/ 

020 

060 

0 

021 

061 

1 

022 

062 

2 

023 

063 

3 

024 

064 

4 

025 

065 

5 

026 

066 

6 

027 

067 

7 

030 

070 

8 

031 

071 

9 

032 

072 

■ 

■ 

033 

073 

■ 

9 

034 

074 

< 

035 

075 

— 

036 

076 

> 

037 

077 

? 

100  §  140  x 

101  A  '  141 

102  B  142 

103  C  143 

104  D  144 

105  E  145 
10G  F  146 
107  G  147 

110  H  150 

111  I  151 

112  J  152 

113  K  153 

114  L  154 

115  11  155 

116  N  156 

117  0  157 

120  P  360 

121  Q  161 

122  R  162 

123  S  163 

124  T  164 

125  U  165 

126  V  166 

127  W  167 

130  X  170 

131  Y  171 

132  Z  172 

133  [  173 

134  -  174 

135  ]  175 

136  a  17G 

137  _  177 


Table  A. 4  GSYMBL  Symbols  and  Corresponding  Octal  1C0DES 
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Implementation 


a.  Subroutines 

The  GS-15  subroutines  consist  of  those  intended  for  users  (e.g.  , 

GMOVE ,  GTEXT,  etc.)  and  those  written  as  internal,  supportive  routine 

All  u.er  subroutines  are  written  in  FORTRAN  IV  and  named  according  to 

a  standard  convention  (the  first  letter  of  the  subroutine  name  is  a 

V').  All  internal  routines  are  written  in  assembly  language  (MACRO- 
15). 


i.  User  Subroutines:  Since  the  user  routines  tend  to  be  very 
short  FORTRAN  subroutines  (typically  less  than  ten  statements), 
thei-  operation  can  easily  be  determined  by  looking  at  the  appro¬ 
priate  source  listing.  Generally  they 

(1)  verify  the  validity  of  parameters, 

(2)  perform  minor  calculations, 

(3)  queue  an  I/O  request  node  to  GLUN  (see  GQUEUE  below),  and 

(4)  return  to  the  caller. 

Any  error  status  is  returned  through  the  "event  variable"  (if 

specified).  The  POOL— currently  a  fifty  word  FORTRAN  vector— 

is  declared  in  the  subroutines  in  a  labeled  common  block  named 
"POOL". 

ii.  Internal  Subroutines:  GS-15  uses  three  internal  subroutines 
that  are  not  visible  to  the  programmer.  Two  of  these  are  general 

purpose  routines  (SETNXT  and  NUMARG)  and  the  other  only  useful 
within  GS-15  (GQUEUE). 


GQUEUE' s  sole  purpose  is  to  take  from  three  to  3ix  of  its 


actual  parameters,  build  them  into  a  standard  RSX  I/O  request  node, 
and  queue  them  to  the  logical  unit  number  specified  in  the  first 
POOL  element  (GLUN).  It  also  zeros  all  unused  I/O  request  node 
entries. 

SETNXT  is  actually  two  routines.  The  first  routine,  SETCHR, 
sets  up  variables  that  will  later  be  used  by  its  companion  routine 
NXTCHR,  an  integer-valued  function.  SETCHR  is  invoked  as  follows: 

CALL  SETCHR  (BUF(l))  . 

BUF  is  a  real  or  double  integer  array  of  packed  5/7  ASCII  text 
(i.e  ,  five  7-bit  characters  per  two  PDP-15  words'.  SETCHR  simply 
defines  the  start  of  a  5/7  ASCII  character  string. 

The  calling  sequence  for  the  function  NXTCHR  is 
NXTCHR (0)  . 

Ncte  that  is  must  be  specified  with  one  parameter  even  though  this 
parameter  is  ignored.  Each  call  to  NXTCHR  returns  the  next  ASCII 
character  in  the  string  defined  by  SETCHR. 

The  last  internal  routine  used  within  the  GS-15  subroutines 
is  an  integer  value  function,  NUMARG.  NUMARG  returns  the  number 
of  actual  arguments  specified  in  the  call  to  the  subroutine  in 
which  NUMARG  appears.  It  is  called  with  one  parameter — the  first 
formal  parameter  of  the  subroutine  that  calls  NUMARG.  NUMARG  is 
undefined  for  subroutines  called  with  zero  actual  parameters.  For 
example,  the  following  subroutine,  TEST,  prints  out  the  number  of 
actual  parameters  in  its  call: 
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SUBROUTINE  TEST(A,B,C,D,E,F,G,H,  I ,  J,K) 

C 

N  =  NUMARG (A) 

PRINT(16 , 1 )N 
1  FORMAT(15) 

RETURN 

END. 

b.  Device  Handlers 

As  mentioned  previously,  GS-15  currently  contains  tvo  device 
handlers  (TV  and  PL).  Following  is  a  discussion  of  the  CRT  handler, 

TV.  Since  these  handlers  are  almost  identical,  a  separate  description 
of  PL  will  not  be  given. 

i.  I/O  Functions:  TV  recognizes  five  standard  RSX  and  five  GS- 
15  I/O  function  codes.  The  RSX  functions  performed  by  TV  include 
ABORT  (01700),  ATTACH  (02400),  DETACH  (02500),  HINF  (03600),  and 
EXIT  (77700).  Please  refer  to  Chapter  Five  of  the  RSX  Reference 
Manual  for  an  explanation  of  these  standard  function.!. 

Since  TV  required  1/0  functions  not  present  in  any  other 
device  handler,  new  function  codes  were  created.  BEGIN  (20000) 
initializes  the  handler  for  a  new  graphics  job;  END  (20100)  per¬ 
forms  some  clean  up  procedures  when  a  task  is  finished  with  the 
handler.  The  connection  between  these  function  codes  and  the 
GBEGIN/GEND  subroutines  should  be  clear.  Similarly,  NEXT  (20400) 
implements  the  GNEXT  subroutine. 

MOVE  (20200)  is  one  of  the  most  important  functions.  A  MOVE 
node  is  queued  for  each  call  on  GM0VE.  The  node  contains  the 
destination  and  pen  position  (UP/DOWN)  of  a  pen  movement.  A 
SYMBOL  (20300) node  is  queued  by  both  GSYMBL  and  GTEXT.  Given  a 
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location,  character,  and  character  offset,  SYMBOL  will  draw  the 
symbol  according  to  the  character  size  and  orientation  currently 
in  effect. 

ii.  Operation:  When  TV  is  assigned  to  a  LUN,  the  handler  does 
some  system-dependent  initialization  (as  distinct  from  BEGIN 
above)  and  then  waits  for  a  node  to  be  queued.  Upon  receiving  a 
node,  TV  jumps  to  an  appropriate  section  based  on  the  I/O  function 
contained  within  the  node.  At  the  completion  of  the  requested 
function,  the  user's  event  variable  is  set,  the  node  is  removed 
from  the  pending  request  queue,  and  the  next  node  is  processed 
(if  one  is  present).  TV  issues  a  system  EXIT  directive  (thus 
removing  itself  from  the  partition)  when  the  user  no  longer 
needs  the  handler  (i.e. ,  the  handler  is  REASSIGNED  through  MCR) . 

TV  uses  three  major  subroutines  to  implement  the  non-standard 
I/O  functions.  XYMOVE  determines  which  elementary  pen  movements 
are  required  to  move  the  pen  to  a  new  position.  This  subroutine 
is  based  on  ACM  Algorithm  162  (an  ALGOL  procedure)  and  calculates 
pen  movement  without  the  use  of  division  or  multiplication. 

Each  of  the  ten  possible  elementary  graphics  commands  (pen 
up/down,  +X,  +X+Y,  etc.)  is  assigned  a  unique  octal  code.  This 
code  is  passed  t  ■>  the  PLOT  subroutine  which  issues  the  actual 
device-dependent  IOT  (I/O  instruction). 

SYMBOL  is  the  device  handler  equivalent  of  GSYMBL.  >t  is 
driven  by  a  character  table  of  approximately  400  words.  Each 

O 

character  in  the  GS-15  character  set  (see  Table  A.  4)  is  defined  by  a 
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APPENDIX  B:  DIGIT--A  DIGITIZING  SYSTEM 


This  program  was  developed  as  a  utility  program  to  digitize 
analog  data  (on-line  or  from  analog  tape)  and  record  it  on  digital 
magnetic  tape  for  off-line  analysis.  The  present  version  is  a  revision 
whica  takes  advantage  of  modifications  in  the  analog-to-digital  conver¬ 
sion  hardware  to  allow  faster  sampling  rates. 

The  program  uses  an  overlay  structure,  defined  as  follows: 

Resident  (Root)  Section:  Executive 
Segment  D1 :  Input  of  data  from  the  operator 
17:  High  speed  data  collection 
18:  Ix>w  speed  data  collection 
D2:  Termination  of  data  gathering  and  closing 
of  tape  records 

D3:  Multiple  file,  termination  of  current  data 
collection  only,  tape  not  closed 

An  added  capability  for  this  version  is  a  timer  which  allows  data  to  be 
taken  for  a  preset  interval  of  time.  After  DIGIT  is  requested  by  the 
user,  Segment  D1  is  called.  It  performs  initialization  and  reallocates 
a  partition  for  use  by  the  actual  data  collection  routine.  Data  for 
an  identification  file  is  input  either  from  the  card  reader  or  from 
the  teletype,  and  the  header  file  is  written  on  the  output  tape.  When 
complete,  the  data  acquisition  segment  (17  or  IS)  is  run.  The  18 
(low  speed)  uses  a  5  msec  data  point  interval.  The  17  (high  speed) 
uses  the  A/D  clock  for  timing,  This  value  may  be  set  by  varying  the 
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clock.  The  nominal  value  is  1  msec,  (requires  f  clock  =  2  KHZ).  This 
task  inputs  data  via  the  A/D  converter,  and  outputs  to  magnetic  tape, 
using  buffered  output.  Following  completion  of  data  collection  (as 
indicated  by  the  user  setting  ACS  0  or  timer  runout),  segment  D2  is 
run.  This  'vegment  runs  a  brief  tape  verification,  printing  the  header 
record  and  scanning  for  record  numbering  errors  as  an  indication  of 
data  loss.  If  multiple  files  are  to  be  written  on  a  single  tape,  the 
termination  is  signified  by  the  user  setting  ACS  1  instead  of  ACS  0 
(this  is  indicated  for  timer  runout  by  setting  ACS  17).  Segment  D3  is 
then  called  to  perform  file  heading  functions. 


USAGE: 

1.  Verify  DIGIT  is  installed. 

2.  Verify  the  device  assignments  shown  below. 


MCR>DEV 
DK0  1 

TT0 _ 2,_3,  4,  12,  13,  14,  20,  21 

RF0  5,10,11,15,17,16 

DT0 

DTI 

MT0  19 

LP0  16 

jCD0  22 

PR0 
FP0 
VP  0 
RD0 
PL0 

_XX& _ _ _ 
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3.  The  following  sequence  shows  reassign  commands  often  required, 
and  the  program  operation  using  the  card  input  of  header  data. 

MCR>REA  19  MT  DT 
MCR>REA  23  NOME  PL 
MCR>REA  22  CD  NONE 
MCR>REQ  DIGIT] 

_PflIA. INPUT  FROM  CD  OR  TT* 

CD 

ENTER  RUNTIME  IN  SECONDS.  B  INDICATES  MANUAL 
STOP  ONLV 
20 

CR  WHEN  READV  TO  BEGIN 

SEJ..ACS  0  EOR  MANUAL  CLOSE/  LAST  FILE 

SET  ACS  1  FOR  MANUAL  CLOSE , CURRENT  FILE  ONLV 

SET  ACS  17  TO  ENABLE  MULTIPLE  FILES  WHEN  USING 

TIMED  EXECUTION  OPTION 

♦♦•DIGITIZING  ACTIVE*** 

-  SXCLP-0 02222-DIGIT 


4.  The  format  of  the  printout  is  shown  below. 


MEAOE*  FILE  CONTENTS 


M/NM  e.o!!"  l5,S''  ”  RH  U,lt  ”  3'»  ‘*1*“*i - 

~'utl  “,t£  111  **'• 

COMMENTS  3LI0E  3  - - - 

_HEC0RQ  COUNT  THIS  TAPE  ■  E4 


5.  Card  format  is  as  follows: 


NMt  Z 
IRIG 
EMMY 
EMMX 
N 

4 

SLIDE  3 
DIANNE  CRHSPY 
3  14:  00:  00 

1 


(CARD  10) 
(CARD  9) 
(CARD  3) 
(CARD  7) 

(CARD  6) 

<CARD  5) 

(CARP  4) 

<CARD  3) 

(CARD  2) 

(CARD  1) 


ooooooooooooooo  0000000000000000000000000000000200000000000000000000C00000 

t  i  i  *  i  i  )  i  ittnntJMBanaannnnMiinnMliNiinaMisiinanMtitMiMOMti  M«tMiiuuMHHi)MHuiiuuHiiMi)wiin  ii  n  n 
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6.  Tape  format 

The  tape  format  is  as  follows: 

Tape  recording  method:  9  track,  800  bpi,  industry 

standard  NRZI  coding. 
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reel  size:  2400  feet 


number  of  files:  2 

The  first  file  contains  5  records  of  81  characters  each.  The  contents 
are  as  shown  under  printout  (4)  above.  The  last  character  on  each 
line  is  a  non-printing  character  (line  feed),  coded  in  EBCDIC  format. 
The  file  is  followed  by  an  end-of-file  mark. 

The  second  file  consists  of  a  variable  number  of  256  word  records. 
The  number  is  determined  by  the  length  of  time  the  digitizing  program 
is  run,  and  is  shown  in  the  tape  verification  printout  (4)  as 
"RECORD  COUNT  THIS  TAPE". 


The  first  four  words  of  each  record*  are  reserved  for  sequence 
number  and  time  information.  Currently,  the  time  data  is  not 
implemented;  this  will  occur  after  the  time  code  generator  interface  is 
operational.  The  remaining  252  words  contain  the  binary  values  of 

A  to  D  converter  output,  in  groups  of  4  channels*.  The  word  format 
is  as  follows: 


PP 

0000 

DATA 

1 

2  5 

6  17 

PP  =  parity  bits 

71,18  f0rmat  appears  when  the  d*ta  is  read  in  using  the  system  direc¬ 
tive  MTGET. 

A  system  library  routine  (RIPOFF)  is  available  to  convert  the 


*  When  using  eight  channels,  the  first  8  words  are  reserved,  and  the 
remaining  248  words  are  in  groups  of  8  channels. 


| 

I 
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above  format  to  a  FORTRAN  compatible  single  integer  word.  For  use 


with  other  operating  systems,  the  data  appears  as  two  consecutive 
8-bit.  bytes  (or  a  16-bit  halfword),  format  as  above  except  the  parity 
bits  are  omitted. 


7.  Teletype  input  of  header  data 

If  teletype  input  is  selected  instead  of  cardreader,  the 
following  sequence  occurs: 


DRTB  INPUT  FROM  CD  OR  TT? 

TT 

ENTER  RECORD  NO 
1 

ENTER  DATE  AND  TIME  RECORDED 
6/2? /?i  13:50:00 
ENTER  SUBJECT  INFORMATION 
DIANNE  CROSBV 

_ENI£&  .COHNENIS _ 

SLIDE  3 

ENTER  NO  OF  CHANNELS  -  4  OR  6 

4 

HIGH  SPEED  VERSION  ?  V  OR  N 
N 

ID  FOR  CHANNEL  _ 1 
'  EMHX 

ID  FOR  CHANNEL  2 
EMMV 

ID  FOR  CHANNEL  3 
IRIG 

ID  FOR  CHANNEL  4 
NONE 


Each  input  is  terminted  by  a  CR.  All  D-numbers  will  have  only  the 
number  entered  (i.e.  ,  do  not  enter  D-9;  enter  9). 
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APPENDIX  C:  MISCELLANEOUS  UTILITY  SOFTWARE 


1.  CORE/DISK  DUMP 

An  RSX  user  may  examine  the  contents  of  the  disk  or  memory  through 
use  of  the  MCR  function,  OPEN.  OPEN  allows  the  inspection  of  only  one 
word  per  invocation;  however,  it  is  unsuited  for  displaying  large, 
contiguous  blocks  of  core  and  disk.  For  this  reason,  an  additional 
MCR  function,  DMP,  was  written.  The  syntax  definition  of  the  DMP 
command  is  as  follows: 

DMP  [D]  from  [  to] 

1—1  L_, 

where  from  =  starting  core/disk  address 
to  s  ending  core/disk  address 
from  a  to. 

Both  the  starting  and  ending  addresses  must  be  octal  constants.  The 
optional  parameter,  "d",  is  specified  when  a  disk  dump  rather  than  a 
core  dump  is  required.  The  dump  listing  will  be  written  on  logical 
unit  LUN  16  which  is  normally  assigned  to  the  line  printer.  DMP  first 
extracts  the  starting  and  ending  addresses  from  the  command  line,  and 
then  initializes  itself  by  filling  the  output  buffer  with  blanks.  As 
each  word  is  received  from  core  or  disk,  it  is  placed  into  the  output 
buffer.  When  the  buffer  is  full  (eight  words),  it  is  printed  on  LUN 
16.  This  procedure  continues  until  the  request  area  is  dumped.  The 
flowchart  for  DMP  is  shown  in  Figure  C.l. 


2.  DEC tape  Dump 


A  program  (DTDUMP)  is  available  to  DUMP  in  octal,  selected  blocks 
of  a  DECtape  assigned  to  LUN  19.  Since  DTDUMP  is  a  task  rather  than  an 
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MCR  function,  its  execution  is  intiated  urough  the  REQUEST  function 
of  RSX.  The  user  is  asked  to  specify  the  beginning  and  ending  blocks 
to  be  dumped  after  which  a  listing  is  produced  on  LUN  16  (line  printer). 
The  program  re-cycles  until  a  starting  block  of  zero  is  given  in 
response  to  DTDUMP's  question. 

3.  Magnetic  Tape  Dump 

This  is  a  system  utility  program  written  to  print  out  the  data 
on  9-track,  800  bpi  industry  compatible  magnetic  tape,  in  an  octal 
format.  It  is  a  task,  called  by  the  system  REQUEST  function.  The 
program  uses  the  console  teletype  for  user  input  of  the  expected  record 
size  and  the  number  of  records.  Each  record  is  read  in,  chi  format 
converted,  and  dumped  to  the  printer.  Printing  suppression  is  used 
for  all  zero  lines.  The  program  will  run  even  if  the  incorrect  record 
size  is  specified,  with  the  extraneous  values  suppressed.  Termination 
occurs  on  end  of  file,  or  completion  of  the  required  number  of  rc  ~-'rds. 
Nc  rewind  is  built  in  to  the  program  to  allow  restarting  the  dump  after 
a  known  number  of  records,  or  after  some  significant  event  on  the  tape 
determined  by  a  preceding  program.  A  brief  flowchart  is  shown  in 
Figure  C.2. 


4.  LAS/ISWTCH 

The  PDP-15  has  a  group  of  eighteen  "data  switches"  or  "sense 
switches  (numbered  0,  1,...,17)  mounted  on  the  operator's  console. 
Since  the  positions  of  these  switches  may  be  dynamically  determined 
through  the  execution  of  a  hardware  instruction  (UAS),  programs  may 
use  these  switches  for  low  volume  data  input.  The  standard  DEC  soft- 
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Figure  C.2  Flowchart  lot  M'i'DUMP  Program 


ware  allows  only  assembly  language  programmers  to  determine  the  data 
switch  settings.  In  order  to  extend  this  capability  for  FORTRAN 

programmers,  two  FORT TAN  callable  routines  have  been  created:  LAS 
and  ISWTCH. 

The  eighteen  data  switches  may  be  thought  of  as  a  single 
eighteen-bit  binary  integer  since  each  switch  must  be  in  only  one  of 
two  possible  positions.  When  a  user  wants  to  ascertain  the  settings  of 
all  or  most  of  the  switches,  one  may  include  the  INTEGER  FUNCTION  LAS 
in  his  program.  When  calling  LAS,  an  argument  must  be  specified.  For 
example,  the  following  short  FORTRAN  program  will  print  the  current 
status  of  the  data  switches  as  a  six-digit  octal  integer: 

N  =  LAS (0) 

PRINT  (13,1)  N 
1  FORMAT  (IX, 06) 

END 

Frequently,  a  program  contains  a  number  of  optional  features  that 
the  user  must  selectively  request.  A  simple  way  to  tell  this  type  of 
program  what  options  are  needed  is  to  assign  one  data  switch  per  option. 
In  order  to  allow  FORTRAN  programs  to  test  single  data  *witche.i,  a 

LOGICAL  FUNCTION  ISWTCH  was  written.  This  routine  is  referenced  as 
follows: 

ISWTCH(I)  where  I  =  INTEGER  variable  or  constant 

(Oils  17) 

ISWTCH(I)  returns  a  "TOTO"  value  it  and  only  if  data  switch  'V  is  on. 

*  FORTRAN  statement  that  would  pause  whenever  both  data  switches  #0  and 
#6  were  on  is  coded  as 


IF  (ISWTCH(O)  .AND.  ISWTCH(6))  PAUSE. 


A  flowchart  depicting  ISWTCH' s  operation  is  given  in  Figure  C.3. 


Figure  C.  3  Flowchart  for  ISWTCH  Program 


5.  RIPOFF 

This  subroutine  was  written  to  convert  data  recorded  on  magnetic 
tape,  or  analog-to-digital  converted  data  to  a  format  consistent  with 
the  internal  formats  for  data  values.  Certain  magnetic  tape  input/ 
output  routines  pass  the  data  to  the  calling  program  without  removing 
the  parity  bits.  Also,  A/D  converter  data  is  12-bits  wide.  Thus, 
negative  values  are  not  properly  represented  in  an  18-bit  word.  This 
subroutine  corrects  for  both  problems.  It  is  currently  in  use  as  a 
FORTRAN  callable  library  routine. 

6.  REA 

The  RSX  operating  system  utilizes  core  partitions  outside  of  the 
monitor  for  input-output  device  handlers.  The  use  of  these  partitions 
is  not  allowed  by  other  tasks  if  the  device  is  on  line  and  in  use  by 
the  system.  This  rubroutine  was  developed  to  dynamically  reallocate 
the  core  space  used  by  these  handlers  by  reassigning  the  device  off 
line.  This  allows  the  use  of  a  peripheral  device  (such  as  a  line 
printer)  to  be  deferred  during  real  time  operations  (such  as  -digitizing) , 
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then  returned  for  use  during  a  following  off-line  operation,  under 

program  control.  It  is  a  FORTRAN  callable  routine,  currently  used  in 
the  DIGIT  pregram. 

7.  GT132 

One  other  special  purpose  su’, routine  is  included  in  the  disk- 
resident  user  library.  The  routine  was  created  because  of  an  early 
need  to  read  digitized  EEG  magnetic  tapes.  Normally  these  tapes  would 
have  been  read  by  the  standard  FORTRAN  input  statements.  However,  at 
the  time  these  tapes  were  needed,  a  FORTRAN  program  could  not  read  them 
•jecause  of  errors  in  the  hardware  and/or  software. 

Therefore  a  subroutine,  GT132,  was  written  which  would  read  one 
record  (132  characters  long)  from  a  digitized  EEG  tape  and  pas  *  this 
record  to  the  user.  The  calling  sequence  for  GT132  is 
CALL  GT132 (BUF.EOF) 

where  BUF  =  27  element  DOUBLE  INTEGER  array 
EOF  =  FORTRAN  statement  number  . 

Each  call  to  GT132  places  a  tape  record  into  BUF  unless  an  end-of-file 
is  encountered.  In  the  latter  case,  GT132  returns  control  to  the 

statement  specified  in  EOF  instead  of  executing  a  normal  RETURN  state¬ 
ment. 

As  shown  in  the  flowchart  in  Figure  C.4,  GT132  translates  the 
EBCDIC  characters  on  tape  into  the  ASCII  character  equivalents  required 
by  RSX  after  first  reading  the  tape  in  binary.  DEC  just  recently 
corrected  the  problem  so  that  we  are  now  able  to  read  the  tapes  with 
the  FORTItAN  READ  statement. 


102 


Figure C.  4  Flowchart  for  GT132  Program 

8.  EMMXY 

The  current  version  of  EMMXY  plots  X  vs.  Y  values  from  stcndard 
digitized  tapes  (see  Appendix  B)  of  eye-movement  data.  The  calibration 
record  for  the  data  is  assumed  to  be  on  disk. 

The  user  is  queried  as  to  the  channel  for  X  data  (1  to  4),  the 
channel  for  Y  data  (1  to  4)  ,  and  the  compression  (compression  is  defined 
as  the  number  of  points  to  skip  be  ween  plotting  points;  it  is  a  method 
of  editing  out  redundant  data  points).  The  program  goes  into  a  wait 
mode  whenever  data  switch  0  is  set  to  a  1  state,  and  exits  on  end  of 
tape  or  whenever  data  switch  1  is  set  to  a  1  state. 

This  is  a  baseline  program.  Several  version  of  this  program  are 
used  by  various  individuals  on  the  project. 

Usage:  (Veiify  LUN-23  -  PLO) 

MCR  >  REQ  EMMXY 


ENTER  COMPRESSION 


ENTER  CHANNEL  NO  -  X 


ENTER  CHANNEL  NO  -  Y 
2 

9.  CALDT 

This  program  allows  the  current  calibration  record  (disk  file 
"CALRC  SRC")  to  be  written  on  DECtape  unit  0  with  a  user  selected  file 
name  (extension  SRC").  The  default  name  is  "CALRC". 

Usage:  (Verify  LUN-19  =  DTO) 

MCR  >  REQ  CALDT 
ENTER  FILE  NAME 

XXXXX  User  inputs  file  name,  5  characters 

maximum.  To  select  default,  CR  only. 

STOP  -  2222  -  CALDT 

10.  CALTD 

This  program  retrieves  a  calibration  record  previously  stored  on 
DECtape  and  prints  the  values.  It  may  then  be  stored  on  disk  as  the 
current  calibration  record  for  analysis  programs. 

Usage:  (Verify  LUN-19  =  DTO) 

MCR  >  REQ  CALTD 
ENTER  FILE  NAME 

XXXXX 

FILE  TO  DISK?  Y  OR  N 
Y 

STOP  -  2222  -  CALTD 
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11.  CALPLT 


This  program  allows  plotting  of  the  calibration  point  matrix. 

The  scaled  values  of  calibration  data  foi  each  point  may  then  be  over- 
layed  from  the  current  calibration  record.  The  pen  may  be  positioned 
in  a  position  (0,0)  to  allow  following  data  to  be  directly  overlayed, 
or  the  pen  may  be  positioned  so  that  subsequent  data  will  not  over¬ 
lay  the  current  record. 

Usage:  (Verify  LUN-23  =  PL  and  LUN-22  /  CD) 

MCR  >  REQ  CALPLT  The  matrix  is  plotted 

CAL  DATA  ALSO?  Y  OR  N 

Y 

POSITION  PEN  FOR  OVERLAY?  Y  OR  N 

Y 

STOP  -  2222  -  CALPLT 

12.  CALCHK 

This  program  accesses  the  current  calibration  record  on  disk  and 
prints  the  values.  No  modification  is  made  to  the  record. 

Usage:  (Verify  LUN-16  =  LP) 

MCR  >  REQ  CALCHK 
STOP  -  2222  -  CALCHK 
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APPENDIX  D:  APPLICATIONS  SOFTWARE 
1-  Eye-Movement  Data  Acquisition  and  Analysis 

a.  Introduction 

This  section  describes  the  programs  designed  for  use  in  acquisition 
and  analysis  of  the  eye -movement  data.  It  is  desirable  to  standardize 
the  operation  of  programs,  in  so  far  as  practical,  to  obtain  the 
maximum  usage  from  any  given  software  "investment".  Thus  a  similar 
approach  will  be  taken  for  EEG  data. 

b.  Data  Flow 

The  software  flow  of  data  under  this  processing  scheme  is  as 
fol lows : 

(1)  A  calibration  progran  !s  run  to  complete  an  overall  system 
calibration.  In  the  case  of  eye-movement  data,  this  involves 
positioning  the  subject's  eyes  at  predetermined  locations.  For 

dimensionless'  measurements,  such  as  EEG,  a  voltage  check  will 
he  made. 

(2)  All  signifi.art  data  are  written  to  the  disk  at  the  time 
of  calibration.  This  allows  subsequent  analysis  programs  to 
access  the  data.  It  also  protects  against  system  crashes  during 
following  real-time  programs  (if  the  data  w re  in  core  it  would 
be  lost). 

(3)  Measurements  are  made  using  the  current  version  of  the 
DIGIT  program.  The  data  may  be  entered  directly,  or  via  analog 
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magnetic  tape.  A  su  adard  format  is  used  for  the  digitized  data 
tape,  including  an  identification  file  and  a  data  file. 


(4)  Analysis  programs  may  then  be  run.  Examples  are  plotting 
and  computational  programs. 

(5)  Utility  programs  are  used  for  transfer  of  disk-stored  data. 

(6)  Manual  procedures  are  required  for  data  logging  (i.e.,  tape 
and  record  identification  numbers). 

c.  Program  Description 

This  section  contains  a  brief  explanation  of  the  calibration 
program  (EMMCL5) .  It  is  basically  user  oriented  and  is  the  current 
release  of  the  eye-movement  monitor  calibration  program.  The  program 
contains  a  prestored  sequeice  for  the  9-point  calibration  chart.  The 
subject  is  directed  to  fixate  on  the  points  as  directed  on  the  console 
teletype.  Following  all  data  gathering,  the  full  scale,  zero  shift 
and  cross  coupling  terms  are  calculated.  The  plot  capability  involves 
calculating  scale  factors  for  plotting  and  writing  these  values  to 
the  disk.  If  immediate  plotting  is  selected,  the  Cal  matrix  and 
corrected  values  are  plotted  (otherwise  this  may  be  done  at  any  time 
up  to  the  next  Cal  by  utility  program  CALPLT) .  An  additional  capability 
is  to  input  data  without  using  the  prestored  sequence.  In  this  case, 
only  scaling  to  volts  is  performed  -  no  attempt  is  made  to  calculate 
the  normal  Cal  terms. 
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usage:  Via  the  GS-15  graphics  system,  the  plot  ou  put  may  be 
on  a  CRT  or  the  CalComp  plotter.  This  program  uses  an  overlay 
structure  to  perform  its  functions  within  core  constraints. 

The  segments  are  as  follows: 

Resident  (Root)  Section:  Calling  or  executive  only. 
Segment  A:  Input  of  data  from  the  operator; 

input  of  eye-movement  measurements. 
Segment  B:  Manipulation  of  the  data  and  scaling. 
Segment  C:  Calculation  and  plotting;  disk  storage 
of  data. 

Verify  LUN  23  is  assigned  to  PLO. 

Verify  correct  data  and  time  via  DATE  function. 

MCP  >  REQ  EMMCL5 


(1)  ENTER  SUBJECT  ID 
xxxxxxxxxx 


10  characters  maximum,  followed  bv 
CR. 


(2)  SET  UP  EYE  MOVEMENT  MONITOR  AND  CALIBRATE  MANUALLY  NL 
WHEN  READY  - 


(3)  PRINT  RAW  DATA?  Y  OR  N 


(4)  PLOT  DATA?  Y  OR  N 


Issue  CR  when  ready 


Followed  by  CR.  This  feature  is 
not  required  for  normal  runs. 


This  selects  immediate  plotting  of 
the  calibration  data. 


(5)  PRESET  CAL  SEQUENCE?  Y  OR  N 


This  selects  the  normal  calibratic 
sequence  for  the  9-point  calibrati 


(6)  DIRECT  SUBJECT  TO  POSITION  PRINTED.  NL  WHEN  READY. 

Note:  The  data  is  taken  for 
each  point  after  the  CR  is  entered. 


At  the  end  of  the  sequence,  the 
summary  data  is  printed.  See  below 
for  format.  The  following  message 
is  printed  if  the  plot  option  is 
chosen. 

(7)  POSITION  PEN  FOR  OVERLAY?  Y  OR  N 

Y  Followed  by  CR.  In  this  case,  the 

pen  is  returned  to  (0,0)  so  that  a 
following  data  analysis  program  may 
overlay  the  plot  on  the  same  co¬ 
ordinate  system.  If  N  is  chosen,  the 
system  goes  to  a  location  to  allow 
following  plots  to  be  made  without 
intef erence. 

(8)  STOP  -  2222  -  EMMCL5  Signifies  end  of  program. 

If  the  (5)  PRESET  CAL  SEQUENCE  is  not  chosen,  the  following 
message  is  printed. 

(9)  ENTER  POSITION,  FOLLOWED  BY  CR.  MAX  OF  23  POINTS 

Note:  as  above,  the  data  is  taken 
after  the  CR.  The  summary  data 
printout  contains  only  the  scaled 
data  values.  If  this  option  is 
chosen,  the  plot  sequence  is  not 
entered,  and  no  data  is  written  to 
the  disk. 
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00)  STOP  -  nil  -  EMMCL5 


Signifies  end  of  program. 


2.  SEER 

We  have  recently  begun  work  on  a  program  that  will  enable  us  to 
do  on-l^ne  analysis  of  visual  methods  of  object  recognition.  SEER, 
unlike  our  earlier  eye  movement  experiments,  is  designed  to  use  the 
Tektronix  611  CRT  for  image  production  and  viewing.  Although  the 

design  is  still  being  developed,  certain  features  have  been  decided 
upon  in  the  SEER  system. 

a.  Operation 

The  most  basic  entity  in  SEER  is  the  "picture".  A  picture 
corresponds  to  a  set  of  data  collected  together  and  given  a  symbolic 
name.  The  data  composing  a  picture  consists  of  an  ordered  set  of 
numbers  which,  w  .en  processed  by  the  GS-15  graphics  system,  produces  an 
image  on  the  surface  of  the  CRT.  These  images  are  then  viewed  by 
subjects  while  SEER  monitors  their  eye  movements,  thus  enabling  us  to 
quickly  determine  scanpaths,  etc. 

We  will  build  up  a  "library"  of  such  pictures  on  DECtape  - 
one  DECtape  file  being  equivalent  to  a  picture.  To  initiate  an 
experiment,  the  picture  library  will  first  be  mounted  on  a  DECtape  unit. 
The  user  must  then  specify  which  pictures  in  the  library  will  be  used. 
These  pictures  are  read  from  DECtape  and  collected  together  in  the 
Picture  "album"  -  a  random  access  disk  file.  Subsequent  SEER  commands 

can  then  refer  to  any  picture  in  the  album  and/or  make  additions  to  or 
doletions  from  the  album. 
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b.  Modularity 

SEER  contains  a  two-level  command  hierarchy.  Major  command 
functions  allow  picture  composition  (CREATE),  picture  viewing  (DR4W) , 
and  album  updating  (ALBUM).  Each  major  command  sub-system  has  its  own 
minor  commands  as  described  in  the  next  section. 

Because  of  the  modular  design  implicit  in  a  major,  minor  command 
structure,  new  features  may  be  easily  added  as  our  experimental 
sophistication  increases.  Furthermore,  since  almost  all  of  tL .  software 
will  be  written  in  FORTRAN  IV,  the  system  could  be  transferred  to 
another  computer  system. 

c.  Major  Sub-Systems 

i.  ALBUM:  Each  picture  in  the  album  has  a  six  character  symbolic 
name  (e.g.,  FACE,  SQUARS ,  etc.).  Pictures  may  be  removed  from 
(DELETE)  or  added  to  (INSERT)  the  album.  A  listing  of  the  album 
(i.e.  ,  which  pictures  are  present)  may  be  obtained  through  use  of 
the  CONTENTS  minor  command.  In  addition,  pictures  in  the  album  may 
be  stored  on  magnetic  tape  by  using  the  SAVE  command. 

ii.  CREATE:  The  purpose  of  this  sub-system  is  to  create  new 
pictures.  Minor  commands  exist  to  "paint"  geometric  objects  such 
as  lines,  rectangles,  and  circles.  Furthermore,  the  TABLET  command 
allows  complex,  non-linear  pictures  to  be  entered  through  a 
writing  table.  Other  commands  NAME  newly  created  pictures  and 
permit  additions  to  pictures  already  in  the  album  (APPEND). 

iii.  DRAW:  In  this  sub-system  pictures  in  the  album  may  be  drawn 
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